[PATCH v4 0/5] Allow better control over container tags

public inbox for isar-users@googlegroups.com
 help / color / mirror / Atom feed

* [PATCH v4 0/5] Allow better control over container tags
@ 2021-08-05 15:57 Henning Schild
  2021-08-05 15:57 ` [PATCH v4 1/5] classes: make sure container extension can run multiple times Henning Schild
                   ` (4 more replies)
  0 siblings, 5 replies; 6+ messages in thread
From: Henning Schild @ 2021-08-05 15:57 UTC (permalink / raw)
  To: isar-users; +Cc: Silvano Cirujano Cuesta, Henning Schild

changed since v3:

- only naming and docs
- rename variables again, patter is "class prefix"+"docker name"
  - docker calls the bits of a tag "TARGET_IMAGE[:TAG]"
  - prefix is "CONTAINER_IMAGE"
- also rename CONTAINER_FORMATS to CONTAINER_IMAGE_FORMATS in p4
- fix docs in p4

changes since v2:

- use CONTAINER_TARGET_IMAGE instead of CONTAINER_IMAGE_NAME
- update docs

changes since v1:

- add RECIPE-API-CHANGELOG entry for interface change in p3

--

A container tags consists of "name:tag" and both have been kind of
hardcoded so far. "tag" was always "latest" which is not proper
versioning. And "name" has always been "isar-DISTRO-ARCH" which is also
not proper because the name should probably describe the content and not
so much the build system. Plus that name means guaranteed collision if
you build two containers with isar based on the same disto and arch.

Henning Schild (5):
  classes: make sure container extension can run multiple times
  classes: simplify tag handling in container class
  classes: allow more control over container image name and tag
  classes: rename CONTAINER_FORMATS to CONTAINER_IMAGE_FORMATS
  classes: fix comment in container-img class

 RECIPE-API-CHANGELOG.md                       |  8 +++++
 doc/user_manual.md                            | 16 +++++-----
 meta/classes/container-img.bbclass            | 10 +++----
 .../classes/image-container-extension.bbclass | 29 ++++++++++---------
 meta/classes/image-sdk-extension.bbclass      |  2 +-
 5 files changed, 36 insertions(+), 29 deletions(-)

-- 
2.31.1


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH v4 1/5] classes: make sure container extension can run multiple times
  2021-08-05 15:57 [PATCH v4 0/5] Allow better control over container tags Henning Schild
@ 2021-08-05 15:57 ` Henning Schild
  2021-08-05 15:57 ` [PATCH v4 2/5] classes: simplify tag handling in container class Henning Schild
                   ` (3 subsequent siblings)
  4 siblings, 0 replies; 6+ messages in thread
From: Henning Schild @ 2021-08-05 15:57 UTC (permalink / raw)
  To: isar-users; +Cc: Silvano Cirujano Cuesta, Henning Schild

If the container extension fails somewhere in the middle, it could leave
things behind resulting in issues when trying to run only that one task
again.

So make sure to remove stuff as root and remove another possibly
existing directory.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta/classes/image-container-extension.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/image-container-extension.bbclass b/meta/classes/image-container-extension.bbclass
index e26604a04f06..6537fc693265 100644
--- a/meta/classes/image-container-extension.bbclass
+++ b/meta/classes/image-container-extension.bbclass
@@ -18,7 +18,7 @@ containerize_rootfs() {
 
     # prepare OCI container image skeleton
     bbdebug 1 "prepare OCI container image skeleton"
-    rm -rf "${oci_img_dir}"
+    sudo rm -rf "${oci_img_dir}" "${oci_img_dir}_unpacked"
     sudo umoci init --layout "${oci_img_dir}"
     sudo umoci new --image "${oci_img_dir}:${empty_tag}"
     sudo umoci config --image "${oci_img_dir}:${empty_tag}" \
-- 
2.31.1


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH v4 2/5] classes: simplify tag handling in container class
  2021-08-05 15:57 [PATCH v4 0/5] Allow better control over container tags Henning Schild
  2021-08-05 15:57 ` [PATCH v4 1/5] classes: make sure container extension can run multiple times Henning Schild
@ 2021-08-05 15:57 ` Henning Schild
  2021-08-05 15:57 ` [PATCH v4 3/5] classes: allow more control over container image name and tag Henning Schild
                   ` (2 subsequent siblings)
  4 siblings, 0 replies; 6+ messages in thread
From: Henning Schild @ 2021-08-05 15:57 UTC (permalink / raw)
  To: isar-users; +Cc: Silvano Cirujano Cuesta, Henning Schild

Call the variable "tag" and use it in all places where "latest" was
hardcoded. This prepares for another patch that will allow chosing
something else than "latest" for a tag.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta/classes/image-container-extension.bbclass | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)

diff --git a/meta/classes/image-container-extension.bbclass b/meta/classes/image-container-extension.bbclass
index 6537fc693265..0e70ba9c1405 100644
--- a/meta/classes/image-container-extension.bbclass
+++ b/meta/classes/image-container-extension.bbclass
@@ -10,7 +10,7 @@ CONTAINER_FORMATS ?= "docker-archive"
 containerize_rootfs() {
     local cmd="/bin/dash"
     local empty_tag="empty"
-    local full_tag="latest"
+    local tag="latest"
     local oci_img_dir="${WORKDIR}/oci-image"
     local rootfs="$1"
     local rootfs_id="$2"
@@ -33,7 +33,7 @@ containerize_rootfs() {
 
     # pack container image
     bbdebug 1 "pack container image"
-    sudo umoci repack --image "${oci_img_dir}:${full_tag}" \
+    sudo umoci repack --image "${oci_img_dir}:${tag}" \
         "${oci_img_dir}_unpacked"
     sudo umoci remove --image "${oci_img_dir}:${empty_tag}"
     sudo rm -rf "${oci_img_dir}_unpacked"
@@ -49,14 +49,14 @@ containerize_rootfs() {
         case "${image_type}" in
             "docker-archive" | "oci-archive")
                 if [ "${image_type}" = "oci-archive" ] ; then
-                    target="${image_type}:${image_archive}:latest"
+                    target="${image_type}:${image_archive}:${tag}"
                 else
-                    target="${image_type}:${image_archive}:${image_name}:latest"
+                    target="${image_type}:${image_archive}:${image_name}:${tag}"
                 fi
                 rm -f "${image_archive}" "${image_archive}.xz"
                 bbdebug 2 "Converting OCI image to ${image_type}"
                 skopeo --insecure-policy copy \
-                    "oci:${oci_img_dir}:${full_tag}" "${target}"
+                    "oci:${oci_img_dir}:${tag}" "${target}"
                 bbdebug 2 "Compressing image"
                 xz -T0 "${image_archive}"
                 ;;
@@ -69,8 +69,8 @@ containerize_rootfs() {
                     die "Adding the container image to a container runtime (${image_type}) not supported if running from a container (e.g. 'kas-container')"
                 fi
                 skopeo --insecure-policy copy \
-                    "oci:${oci_img_dir}:${full_tag}" \
-                    "${image_type}:${image_name}:latest"
+                    "oci:${oci_img_dir}:${tag}" \
+                    "${image_type}:${image_name}:${tag}"
                 ;;
             *)
                 die "Unsupported format for containerize_rootfs: ${image_type}"
-- 
2.31.1


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH v4 3/5] classes: allow more control over container image name and tag
  2021-08-05 15:57 [PATCH v4 0/5] Allow better control over container tags Henning Schild
  2021-08-05 15:57 ` [PATCH v4 1/5] classes: make sure container extension can run multiple times Henning Schild
  2021-08-05 15:57 ` [PATCH v4 2/5] classes: simplify tag handling in container class Henning Schild
@ 2021-08-05 15:57 ` Henning Schild
  2021-08-05 15:57 ` [PATCH v4 4/5] classes: rename CONTAINER_FORMATS to CONTAINER_IMAGE_FORMATS Henning Schild
  2021-08-05 15:57 ` [PATCH v4 5/5] classes: fix comment in container-img class Henning Schild
  4 siblings, 0 replies; 6+ messages in thread
From: Henning Schild @ 2021-08-05 15:57 UTC (permalink / raw)
  To: isar-users; +Cc: Silvano Cirujano Cuesta, Henning Schild

This patch allows more fine-grained control over how the resulting
container will be tagged. Where the default name will be PN together
with DISTRO and ARCH, and tag will be derived from PV and PR

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 RECIPE-API-CHANGELOG.md                        |  4 ++++
 doc/user_manual.md                             |  8 ++++----
 meta/classes/container-img.bbclass             |  4 +---
 meta/classes/image-container-extension.bbclass | 11 ++++++-----
 meta/classes/image-sdk-extension.bbclass       |  2 +-
 5 files changed, 16 insertions(+), 13 deletions(-)

diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 806c2914f13f..a698eafc8843 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -292,3 +292,7 @@ Migrate your patches so they can be applied with "git am", or
 
 Kernel image name for arm64 platforms is vmlinux now. Image format was
 not changed (uncompressed executable) but now it is named correctly.
+
+### Change default "TARGET_IMAGE:TAG" when building container images
+
+The "TARGET_IMAGE" used to be rather static and the TAG was always "latest", now the values are derived from recipe variables PN, PV, PR.
diff --git a/doc/user_manual.md b/doc/user_manual.md
index cf7dc2fee35e..e5c07bd41fc5 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -300,14 +300,14 @@ bitbake mc:qemuarm-buster:isar-image-base
  - Load the container image into the Docker Daemon
 
 ```
-docker load -i build/tmp/deploy/images/qemuarm/debian-buster-armhf-docker-archive.tar.xz
+docker load -i build/tmp/deploy/images/qemuarm/isar-image-base-debian-buster-armhf-1.0-r0-docker-archive.tar.xz
 ```
 
  - Run a container using the container image (following commands starting with 
    `#~:` are to be run in the container)
 
 ```
-docker run --rm -ti --volume "$(pwd):/build" isar-buster-armhf:latest
+docker run --rm -ti --volume "$(pwd):/build" isar-image-base-debian-buster-armhf:1.0-r0
 ```
 
 ---
@@ -1070,14 +1070,14 @@ bitbake -c do_populate_sdk mc:qemuarm-buster:isar-image-base
  - Load the SDK container image into the Docker Daemon
 
 ```
-docker load -i build/tmp/deploy/images/qemuarm/sdk-debian-buster-armhf-docker-archive.tar.xz
+docker load -i build/tmp/deploy/images/qemuarm/isar-image-base-debian-buster-armhf-1.0-r0-docker-archive.tar.xz
 ```
 
  - Run a container using the SDK container image (following commands starting 
    with `#~:` are to be run in the container)
 
 ```
-docker run --rm -ti --volume "$(pwd):/build" isar-sdk-buster-armhf:latest
+docker run --rm -ti --volume "$(pwd):/build" isar-image-base-debian-buster-armhf:1.0-r0
 ```
 
  - Check that cross toolchains are installed
diff --git a/meta/classes/container-img.bbclass b/meta/classes/container-img.bbclass
index 79ef3e8d756b..9928a58ef53d 100644
--- a/meta/classes/container-img.bbclass
+++ b/meta/classes/container-img.bbclass
@@ -9,10 +9,8 @@
 do_container_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
 do_container_image[vardeps] += "CONTAINER_FORMATS"
 do_container_image(){
-    rootfs_id="${DISTRO}-${DISTRO_ARCH}"
-
     bbdebug 1 "Generate container image in these formats: ${CONTAINER_FORMATS}"
-    containerize_rootfs "${IMAGE_ROOTFS}" "${rootfs_id}" "${CONTAINER_FORMATS}"
+    containerize_rootfs "${IMAGE_ROOTFS}" "${CONTAINER_FORMATS}"
 }
 
 addtask container_image before do_image after do_image_tools
diff --git a/meta/classes/image-container-extension.bbclass b/meta/classes/image-container-extension.bbclass
index 0e70ba9c1405..cfbc5f2f3786 100644
--- a/meta/classes/image-container-extension.bbclass
+++ b/meta/classes/image-container-extension.bbclass
@@ -6,15 +6,16 @@
 # This class extends the image.bbclass for containerizing the root filesystem.
 
 CONTAINER_FORMATS ?= "docker-archive"
+CONTAINER_IMAGE_TARGET_IMAGE ?= "${PN}-${DISTRO}-${DISTRO_ARCH}"
+CONTAINER_IMAGE_TAG ?= "${PV}-${PR}"
 
 containerize_rootfs() {
     local cmd="/bin/dash"
     local empty_tag="empty"
-    local tag="latest"
+    local tag="${CONTAINER_IMAGE_TAG}"
     local oci_img_dir="${WORKDIR}/oci-image"
     local rootfs="$1"
-    local rootfs_id="$2"
-    local container_formats="$3"
+    local container_formats="$2"
 
     # prepare OCI container image skeleton
     bbdebug 1 "prepare OCI container image skeleton"
@@ -42,9 +43,9 @@ containerize_rootfs() {
     sudo chown --recursive $(id -u):$(id -g) "${oci_img_dir}"
 
     # convert the OCI container image to the desired format
-    image_name="isar-${rootfs_id}"
+    image_name="${CONTAINER_IMAGE_TARGET_IMAGE}"
     for image_type in ${CONTAINER_FORMATS} ; do
-        image_archive="${DEPLOY_DIR_IMAGE}/${rootfs_id}-${image_type}.tar"
+        image_archive="${DEPLOY_DIR_IMAGE}/${image_name}-${tag}-${image_type}.tar"
         bbdebug 1 "Creating container image type: ${image_type}"
         case "${image_type}" in
             "docker-archive" | "oci-archive")
diff --git a/meta/classes/image-sdk-extension.bbclass b/meta/classes/image-sdk-extension.bbclass
index 426b92595554..fa15b588068c 100644
--- a/meta/classes/image-sdk-extension.bbclass
+++ b/meta/classes/image-sdk-extension.bbclass
@@ -80,7 +80,7 @@ do_populate_sdk() {
     # generate the SDK in all the desired container formats
     if [ -n "${sdk_container_formats}" ] ; then
         bbnote "Generating SDK container in ${sdk_container_formats} format"
-        containerize_rootfs "${SDKCHROOT_DIR}" "sdk-${DISTRO}-${DISTRO_ARCH}" "${sdk_container_formats}"
+        containerize_rootfs "${SDKCHROOT_DIR}" "${sdk_container_formats}"
     fi
 }
 
-- 
2.31.1


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH v4 4/5] classes: rename CONTAINER_FORMATS to CONTAINER_IMAGE_FORMATS
  2021-08-05 15:57 [PATCH v4 0/5] Allow better control over container tags Henning Schild
                   ` (2 preceding siblings ...)
  2021-08-05 15:57 ` [PATCH v4 3/5] classes: allow more control over container image name and tag Henning Schild
@ 2021-08-05 15:57 ` Henning Schild
  2021-08-05 15:57 ` [PATCH v4 5/5] classes: fix comment in container-img class Henning Schild
  4 siblings, 0 replies; 6+ messages in thread
From: Henning Schild @ 2021-08-05 15:57 UTC (permalink / raw)
  To: isar-users; +Cc: Silvano Cirujano Cuesta, Henning Schild

This brings some consistency with other image-classes, that image class
variables are prefixed with that class.

Also fix the docs (where the trailing S was missing), and mention the
interface break in our changelog.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 RECIPE-API-CHANGELOG.md                        | 4 ++++
 doc/user_manual.md                             | 8 ++++----
 meta/classes/container-img.bbclass             | 6 +++---
 meta/classes/image-container-extension.bbclass | 4 ++--
 4 files changed, 13 insertions(+), 9 deletions(-)

diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index a698eafc8843..d60588e55e7e 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -296,3 +296,7 @@ not changed (uncompressed executable) but now it is named correctly.
 ### Change default "TARGET_IMAGE:TAG" when building container images
 
 The "TARGET_IMAGE" used to be rather static and the TAG was always "latest", now the values are derived from recipe variables PN, PV, PR.
+
+### Renamed variable CONTAINER_FORMATS to CONTAINER_IMAGE_FORMATS
+
+The meaning remains the same, just the name changed.
diff --git a/doc/user_manual.md b/doc/user_manual.md
index e5c07bd41fc5..e000a71d47f7 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -255,7 +255,7 @@ requirements (e.g. libraries) can be easily resolved in a containerized
 environment.
 
 Container images can be generated in different formats, selected with the 
-variable `CONTAINER_FORMAT`. One or more (whitespace separated) of following 
+variable `CONTAINER_IMAGE_FORMATS`. One or more (whitespace separated) of following 
 options can be given:
  - `docker-archive`: (default) an archive containing a Docker image that can 
    be imported with [`docker load`](https://docs.docker.com/engine/reference/commandline/load)
@@ -277,7 +277,7 @@ The resulting container image archives (only for `docker-archive` and
 `oci-archive`) are made available as 
 `tmp/deploy/images/${MACHINE}/${DISTRO}-${DISTRO_ARCH}-${container_format}.tar.xz` 
 (being `container_format` each one of the formats specified in 
-`CONTAINER_FORMAT`).
+`CONTAINER_IMAGE_FORMATS`).
 
 ### Example
 
@@ -286,9 +286,9 @@ The resulting container image archives (only for `docker-archive` and
 For one-shot builds (use `local.conf` otherwise):
 
 ```
-export BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE IMAGE_TYPE CONTAINER_FORMAT"
+export BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE IMAGE_TYPE CONTAINER_IMAGE_FORMATS"
 export IMAGE_TYPE="container-img"
-export CONTAINER_FORMAT="docker-archive"
+export CONTAINER_IMAGE_FORMATS="docker-archive"
 ```
 
  - Trigger creation of container image from root filesystem
diff --git a/meta/classes/container-img.bbclass b/meta/classes/container-img.bbclass
index 9928a58ef53d..174502ddca7a 100644
--- a/meta/classes/container-img.bbclass
+++ b/meta/classes/container-img.bbclass
@@ -7,10 +7,10 @@
 # to create container images containing the target rootfs.
 
 do_container_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
-do_container_image[vardeps] += "CONTAINER_FORMATS"
+do_container_image[vardeps] += "CONTAINER_IMAGE_FORMATS"
 do_container_image(){
-    bbdebug 1 "Generate container image in these formats: ${CONTAINER_FORMATS}"
-    containerize_rootfs "${IMAGE_ROOTFS}" "${CONTAINER_FORMATS}"
+    bbdebug 1 "Generate container image in these formats: ${CONTAINER_IMAGE_FORMATS}"
+    containerize_rootfs "${IMAGE_ROOTFS}" "${CONTAINER_IMAGE_FORMATS}"
 }
 
 addtask container_image before do_image after do_image_tools
diff --git a/meta/classes/image-container-extension.bbclass b/meta/classes/image-container-extension.bbclass
index cfbc5f2f3786..8040a991f280 100644
--- a/meta/classes/image-container-extension.bbclass
+++ b/meta/classes/image-container-extension.bbclass
@@ -5,7 +5,7 @@
 #
 # This class extends the image.bbclass for containerizing the root filesystem.
 
-CONTAINER_FORMATS ?= "docker-archive"
+CONTAINER_IMAGE_FORMATS ?= "docker-archive"
 CONTAINER_IMAGE_TARGET_IMAGE ?= "${PN}-${DISTRO}-${DISTRO_ARCH}"
 CONTAINER_IMAGE_TAG ?= "${PV}-${PR}"
 
@@ -44,7 +44,7 @@ containerize_rootfs() {
 
     # convert the OCI container image to the desired format
     image_name="${CONTAINER_IMAGE_TARGET_IMAGE}"
-    for image_type in ${CONTAINER_FORMATS} ; do
+    for image_type in ${CONTAINER_IMAGE_FORMATS} ; do
         image_archive="${DEPLOY_DIR_IMAGE}/${image_name}-${tag}-${image_type}.tar"
         bbdebug 1 "Creating container image type: ${image_type}"
         case "${image_type}" in
-- 
2.31.1


^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH v4 5/5] classes: fix comment in container-img class
  2021-08-05 15:57 [PATCH v4 0/5] Allow better control over container tags Henning Schild
                   ` (3 preceding siblings ...)
  2021-08-05 15:57 ` [PATCH v4 4/5] classes: rename CONTAINER_FORMATS to CONTAINER_IMAGE_FORMATS Henning Schild
@ 2021-08-05 15:57 ` Henning Schild
  4 siblings, 0 replies; 6+ messages in thread
From: Henning Schild @ 2021-08-05 15:57 UTC (permalink / raw)
  To: isar-users; +Cc: Silvano Cirujano Cuesta, Henning Schild

The task name is in fact a different one, the function sits in another
class.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta/classes/container-img.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/container-img.bbclass b/meta/classes/container-img.bbclass
index 174502ddca7a..dce984073d09 100644
--- a/meta/classes/container-img.bbclass
+++ b/meta/classes/container-img.bbclass
@@ -3,7 +3,7 @@
 #
 # SPDX-License-Identifier: MIT
 #
-# This class provides the task 'containerize_rootfs'
+# This class provides the task 'container_image'
 # to create container images containing the target rootfs.
 
 do_container_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
-- 
2.31.1


^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2021-08-05 15:57 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-05 15:57 [PATCH v4 0/5] Allow better control over container tags Henning Schild
2021-08-05 15:57 ` [PATCH v4 1/5] classes: make sure container extension can run multiple times Henning Schild
2021-08-05 15:57 ` [PATCH v4 2/5] classes: simplify tag handling in container class Henning Schild
2021-08-05 15:57 ` [PATCH v4 3/5] classes: allow more control over container image name and tag Henning Schild
2021-08-05 15:57 ` [PATCH v4 4/5] classes: rename CONTAINER_FORMATS to CONTAINER_IMAGE_FORMATS Henning Schild
2021-08-05 15:57 ` [PATCH v4 5/5] classes: fix comment in container-img class Henning Schild

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox