From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6927266035414335488 X-Received: by 2002:a2e:bc0c:: with SMTP id b12mr2288285ljf.201.1613150111980; Fri, 12 Feb 2021 09:15:11 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:651c:1041:: with SMTP id x1ls1895588ljm.4.gmail; Fri, 12 Feb 2021 09:15:10 -0800 (PST) X-Google-Smtp-Source: ABdhPJyLIS3liZQuSsFKvqCHOHTMqr0XByEKX12lVcce0PQuMD/pPim+vakdxIZXgg3MWElpsUym X-Received: by 2002:a2e:9dc3:: with SMTP id x3mr2248333ljj.434.1613150110745; Fri, 12 Feb 2021 09:15:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1613150110; cv=none; d=google.com; s=arc-20160816; b=QsQKfhyNZTTmm26b4AUTtvfiqceofuShdqtaS5vrOWkoIuniCBVKNfR1lnwXVE3rG0 0qilIviF0tJ0esMUwtnmgmUheVJbrH4GmGebNl0wtgS5UI13rGY3o72vAlrlU7v8Y/d0 BMt+m2dn7AOpLLRfNBtj8DB291BsCIFpdDIyyEdmdzD1MHT+fjl2Fkf6qWJ8jzxJLWfm 77KaE9qnbp0QRz9R0WZB3g4Fyl+LyECrl0AVYBWPgLnKsL/YYrHkVd00huiDjyB0EiDz 2v25VxnzYfKlPxXOpWL9VpB9JdTr7skS9SK3zIoZkx+q+9ZeI4ONPO0f6ljaTKw51gC+ dkEg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject; bh=d0tIqCPPBQIJ6ymgO3fUU7M38jVHVoAsgk/PIxQ35vQ=; b=nPe4OyM48wJ2PU2jvEw4Mbpy1YiwbvcHaLK41QJeQv1qYeAcVIuau2XwkqdTVgUUxf zmriXTYmZXOKtoAjygbvZRtwPQ3RTbM01PzcHkOlii8gCukzB4oiX/6Y98bB6Xkr1v6Q oU0yDbQAVSBENytTg3LZtEznzI5L00qoY/OeMUHDiKwno8zmhWO9v/6XGQPQYJqfGyWo Y/fHl9++Knw3IXrJUq4e3IRs4G6jcyiIcV4FwmLGg9tuoJTErlZ1Npl1591O2zJ15Geq fZrqPnKDN2+cGyUSyaGxZ1J/rVYwejA6kad2YN+crtkZnY/byNGUwOIdJu7hwC/7Iz18 thxw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from gecko.sbs.de (gecko.sbs.de. [194.138.37.40]) by gmr-mx.google.com with ESMTPS id w18si582983ljw.1.2021.02.12.09.15.10 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 12 Feb 2021 09:15:10 -0800 (PST) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.40 as permitted sender) client-ip=194.138.37.40; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by gecko.sbs.de (8.15.2/8.15.2) with ESMTPS id 11CHF9MB030541 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Fri, 12 Feb 2021 18:15:09 +0100 Received: from [167.87.240.24] ([167.87.240.24]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 11CHA9wl002433; Fri, 12 Feb 2021 18:10:09 +0100 Subject: Re: [PATCH v3 1/2] images: add support for container images To: "[ext] Silvano Cirujano Cuesta" , isar-users@googlegroups.com References: <20210212085113.11013-1-silvano.cirujano-cuesta@siemens.com> <20210212085113.11013-2-silvano.cirujano-cuesta@siemens.com> From: Jan Kiszka Message-ID: Date: Fri, 12 Feb 2021 18:10:09 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.0 MIME-Version: 1.0 In-Reply-To: <20210212085113.11013-2-silvano.cirujano-cuesta@siemens.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: 2nQLRN8iGVsH On 12.02.21 09:51, [ext] Silvano Cirujano Cuesta wrote: > Add support for creation of container images with the build root > filesystems. > > Extend also task "populate_sdk" to support the creation of a container image > containing the SDK. Should be done in to steps: container-img.bbclass frirst, and then a patch to use it for the SDK as well. > > Signed-off-by: Silvano Cirujano Cuesta > --- > meta/classes/container-img.bbclass | 88 ++++++++++++++++++++++++ > meta/classes/image-sdk-extension.bbclass | 51 ++++++++++++-- > meta/classes/image.bbclass | 1 + > 3 files changed, 133 insertions(+), 7 deletions(-) > create mode 100644 meta/classes/container-img.bbclass > > diff --git a/meta/classes/container-img.bbclass b/meta/classes/container-img.bbclass > new file mode 100644 > index 0000000..35c7bbc > --- /dev/null > +++ b/meta/classes/container-img.bbclass > @@ -0,0 +1,88 @@ > +# This software is a part of ISAR. > +# Copyright (C) Siemens AG, 2021 > +# > +# SPDX-License-Identifier: MIT > +# > +# This class provides the tasks 'containerize_rootfs' and 'containerize_sdk' Nope, it now only provides the former. > +# to create container images containing the target rootfs and the SDK > +# respectively. > + > +CONTAINER_FORMATS ?= "docker-archive" > + > +containerize_rootfs() { > + local cmd="/bin/dash" > + local empty_tag="empty" > + local full_tag="latest" > + local oci_img_dir="${WORKDIR}/oci-image" > + local rootfs="$1" > + local rootfs_id="$2" > + local container_formats="$3" > + > + # prepare OCI container image skeleton > + bbdebug 1 "prepare OCI container image skeleton" > + rm -rf "${oci_img_dir}" > + sudo umoci init --layout "${oci_img_dir}" > + sudo umoci new --image "${oci_img_dir}:${empty_tag}" > + sudo umoci config --image "${oci_img_dir}:${empty_tag}" \ > + --config.cmd="${cmd}" > + sudo umoci unpack --image "${oci_img_dir}:${empty_tag}" \ > + "${oci_img_dir}_unpacked" > + > + # add root filesystem as the flesh of the skeleton > + sudo cp -a "${rootfs}"/* "${oci_img_dir}_unpacked/rootfs/" > + > + # pack container image > + bbdebug 1 "pack container image" > + sudo umoci repack --image "${oci_img_dir}:${full_tag}" \ > + "${oci_img_dir}_unpacked" > + sudo umoci remove --image "${oci_img_dir}:${empty_tag}" > + sudo rm -rf "${oci_img_dir}_unpacked" > + > + # no root needed anymore > + sudo chown --recursive $(id -u):$(id -g) "${oci_img_dir}" > + > + # convert the OCI container image to the desired format > + image_name="isar-${rootfs_id}" > + for image_type in ${CONTAINER_FORMATS} ; do > + image_archive="${DEPLOY_DIR_IMAGE}/${rootfs_id}-${image_type}.tar" > + bbdebug 1 "Creating container image type: ${image_type}" > + case "${image_type}" in > + "docker-archive" | "oci-archive") > + if [ "${image_type}" = "oci-archive" ] ; then > + target="${image_type}:${image_archive}:latest" > + else > + target="${image_type}:${image_archive}:${image_name}:latest" > + fi > + rm -f "${image_archive}" "${image_archive}.xz" > + bbdebug 2 "Converting OCI image to ${image_type}" > + skopeo --insecure-policy copy \ > + "oci:${oci_img_dir}:${full_tag}" "${target}" > + bbdebug 2 "Compressing image" > + xz -T0 "${image_archive}" > + ;; > + "oci") > + tar --create --xz --directory "${oci_img_dir}" \ > + --file "${image_archive}.xz" . > + ;; > + "docker-daemon" | "containers-storage") > + skopeo --insecure-policy copy \ > + "oci:${oci_img_dir}:${full_tag}" \ > + "${image_type}:${image_name}:latest" > + ;; Missing check for "Am I in a container?", like in the SDK. Maybe move that test here and share. > + *) > + die "Unsupported format for containerize_rootfs: ${image_type}" > + ;; > + esac > + done > +} > + > +do_container_image[stamp-extra-info] = "${DISTRO}-${MACHINE}" > +do_container_image[vardeps] += "CONTAINER_FORMATS" > +do_container_image(){ > + rootfs_id="${DISTRO}-${DISTRO_ARCH}" > + > + bbnote "Generate container image in these formats: ${CONTAINER_FORMATS}" Probabably more "bbdebug"? Unsure. But we aren't using bbnote in the core so far. Nor bbdebug, though. > + containerize_rootfs "${IMAGE_ROOTFS}" "${rootfs_id}" "${CONTAINER_FORMATS}" > +} > + > +addtask container_image before do_image after do_image_tools > diff --git a/meta/classes/image-sdk-extension.bbclass b/meta/classes/image-sdk-extension.bbclass > index a8c708a..63138da 100644 > --- a/meta/classes/image-sdk-extension.bbclass > +++ b/meta/classes/image-sdk-extension.bbclass > @@ -6,11 +6,25 @@ > # This class extends the image.bbclass to supply the creation of a sdk > > SDK_INCLUDE_ISAR_APT ?= "0" > +SDK_FORMATS ?= "tar-xz" > + > +sdk_tar_xz() { > + # Copy mount_chroot.sh for convenience > + sudo cp ${SCRIPTSDIR}/mount_chroot.sh ${SDKCHROOT_DIR} > + > + # Create SDK archive > + cd -P ${SDKCHROOT_DIR}/.. > + sudo tar --transform="s|^rootfs|sdk-${DISTRO}-${DISTRO_ARCH}|" \ > + -c rootfs | xz -T0 > ${DEPLOY_DIR_IMAGE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz > + bbnote "SDK rootfs available in ${DEPLOY_DIR_IMAGE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz" > +} > > do_populate_sdk[stamp-extra-info] = "${DISTRO}-${MACHINE}" > do_populate_sdk[depends] = "sdkchroot:do_build" > -do_populate_sdk[vardeps] += "SDK_INCLUDE_ISAR_APT" > +do_populate_sdk[vardeps] += "SDK_INCLUDE_ISAR_APT SDK_FORMATS" > do_populate_sdk() { > + local sdk_container_formats="" > + > if [ "${SDK_INCLUDE_ISAR_APT}" = "1" ]; then > # Copy isar-apt with deployed Isar packages > sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO} ${SDKCHROOT_DIR}/isar-apt > @@ -48,12 +62,35 @@ do_populate_sdk() { > done > done > > - # Copy mount_chroot.sh for convenience > - sudo cp ${SCRIPTSDIR}/mount_chroot.sh ${SDKCHROOT_DIR} > + # separate SDK formats: TAR and container formats > + for sdk_format in ${SDK_FORMATS} ; do > + case ${sdk_format} in > + "tar-xz") > + sdk_tar_xz > + ;; > + "docker-archive" | "oci" | "oci-archive") > + if [ -z "${sdk_container_formats}" ] ; then Unneeded, just use the else part unconditionally. > + sdk_container_formats="${sdk_format}" > + else > + sdk_container_formats="${sdk_container_formats} ${sdk_format}" > + fi > + ;; > + "docker-daemon" | "containers-storage") > + if [ -f /.dockerenv ] || [ -f /run/.containerenv ] ; then > + die "Adding the SDK container image to a container runtime (${sdk_format}) not supported if running from a container (e.g. 'kas-container')" > + fi See above, should likely go into containerize_rootfs(). > + ;; > + *) > + die "unsupported SDK format specified: ${sdk_format}" > + ;; > + esac > + done > > - # Create SDK archive > - cd -P ${SDKCHROOT_DIR}/.. > - sudo tar --transform="s|^rootfs|sdk-${DISTRO}-${DISTRO_ARCH}|" \ > - -c rootfs | xz -T0 > ${DEPLOY_DIR_IMAGE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz > + # generate the SDK in all the desired container formats > + if [ -n "${sdk_container_formats}" ] ; then > + bbnote "Generating SDK container in ${sdk_container_formats} format" > + containerize_rootfs "${SDKCHROOT_DIR}" "sdk-${DISTRO}-${DISTRO_ARCH}" "${sdk_container_formats}" > + fi > } > + > addtask populate_sdk after do_rootfs > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass > index eddc444..7fb7b7e 100644 > --- a/meta/classes/image.bbclass > +++ b/meta/classes/image.bbclass > @@ -76,6 +76,7 @@ inherit image-tools-extension > inherit image-postproc-extension > inherit image-locales-extension > inherit image-account-extension > +inherit container-img > > # Extra space for rootfs in MB > ROOTFS_EXTRA ?= "64" > Jan -- Siemens AG, T RDA IOT Corporate Competence Center Embedded Linux