From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6925703954041929728 X-Received: by 2002:a17:906:40f:: with SMTP id d15mr3667092eja.522.1612524588545; Fri, 05 Feb 2021 03:29:48 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6402:1432:: with SMTP id c18ls6047974edx.0.gmail; Fri, 05 Feb 2021 03:29:47 -0800 (PST) X-Google-Smtp-Source: ABdhPJxm8qiXohUD7hM96/VGag8H8E+XV4x2LgbnR/kj2fphOBa5GT2Mmk1sr4TruyKRrf7HxnoP X-Received: by 2002:aa7:cb0d:: with SMTP id s13mr3032247edt.221.1612524587485; Fri, 05 Feb 2021 03:29:47 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1612524587; cv=none; d=google.com; s=arc-20160816; b=yK7jnrO8atGKft++PZCYzo500zsfk3oGn4/mmHhECUGRfTZWYuZz9ejPNlg2VugdOV Sa15uklMZ85sCED7jVguvG4GH1SfarC/sZ1kk9zlg8vmWRMevqqcBY768dlfckni9tZO 4GvQh4Y97l5KYXDK37+Vmr3dkFrrZZZy6qm8/HRTlyYD9eDN4XRHRdzOlSZ/kalo2E0d U/mTH12VDHL0VLGhdQSoJXBWGHkk9F84Vd8Uo9ctXYVDoUhKhdLqIfamIverda1fVbwF YYleqZZDsZ9zCUYet9WPnhVy/iro0Xcu7F/73lwGvIGRYnmwJWvhUBAIyc7bEkDYgk21 rteA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject; bh=7LFWPEFXQAJrVpBa6NsfLdbteXe0NFIfTiQ4YTY1y2s=; b=Ds3N5BbwpBiECi5SlDTXk5LNfIjTQ3Uub430+YESK29oxa4e6Hwqe+WP6IQji6Y8aM UxiSrq+lS3g3D+8tUQ5k1vTfyR/rubhdSctFcTft076Oio+uSZPqMsKhLwMsaWiW+2A8 zjJlk8tsaLJKJztpt5ZSXOww4CBzLPriiXCSIT8MmoWVPbdADOMr8GbFYjlneMdcnoNY hvEwY+jYSXA1eo3fxA6sCQe/oLc+1565SW2WkiOkLWRg/YoVDZNurNeGtC+x6H0zZqXk fz9WnzQiyur5aJRBXgVMrMEAt4BXHpxpATwpLMy4aDTFpB6iuj/Il5+DP8yNcZIKlJda 2lWA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from goliath.siemens.de (goliath.siemens.de. [192.35.17.28]) by gmr-mx.google.com with ESMTPS id a15si474453edn.0.2021.02.05.03.29.47 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 05 Feb 2021 03:29:47 -0800 (PST) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) client-ip=192.35.17.28; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id 115BTl7K018585 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Fri, 5 Feb 2021 12:29:47 +0100 Received: from [167.87.72.79] ([167.87.72.79]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 115BTkY1016339; Fri, 5 Feb 2021 12:29:46 +0100 Subject: Re: [PATCH 1/2] sdk: support creation of container image To: Silvano Cirujano Cuesta , isar-users@googlegroups.com References: <20210205090827.17788-1-silvano.cirujano-cuesta@siemens.com> <20210205090827.17788-2-silvano.cirujano-cuesta@siemens.com> <1bf47211-0313-48a9-00d8-442e6f9942ae@siemens.com> <62b4993c-0a7d-f29d-9c65-978936fd8e7c@siemens.com> From: Jan Kiszka Message-ID: <724da05d-7eeb-73de-05a4-5effab51086f@siemens.com> Date: Fri, 5 Feb 2021 12:29:46 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.0 MIME-Version: 1.0 In-Reply-To: <62b4993c-0a7d-f29d-9c65-978936fd8e7c@siemens.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: /O18k+CmWFHj On 05.02.21 12:24, Silvano Cirujano Cuesta wrote: > > On 05/02/2021 12:07, Jan Kiszka wrote: >> On 05.02.21 10:08, [ext] Silvano Cirujano Cuesta wrote: >>> Extend task "populate_sdk" to support the creation of a container image >>> containing the SDK. >>> >>> Signed-off-by: Silvano Cirujano Cuesta >>> --- >>> meta/classes/image-sdk-extension.bbclass | 104 +++++++++++++++++++++-- >>> 1 file changed, 97 insertions(+), 7 deletions(-) >>> >>> diff --git a/meta/classes/image-sdk-extension.bbclass b/meta/classes/image-sdk-extension.bbclass >>> index a8c708a..082b16d 100644 >>> --- a/meta/classes/image-sdk-extension.bbclass >>> +++ b/meta/classes/image-sdk-extension.bbclass >>> @@ -6,10 +6,81 @@ >>> # This class extends the image.bbclass to supply the creation of a sdk >>> >>> SDK_INCLUDE_ISAR_APT ?= "0" >>> +SDK_FORMATS ?= "tar-xz" >>> + >>> +sdk_tar_xz() { >>> + # Copy mount_chroot.sh for convenience >>> + sudo cp ${SCRIPTSDIR}/mount_chroot.sh ${SDKCHROOT_DIR} >>> + >>> + # Create SDK archive >>> + cd -P ${SDKCHROOT_DIR}/.. >>> + sudo tar --transform="s|^rootfs|sdk-${DISTRO}-${DISTRO_ARCH}|" \ >>> + -c rootfs | xz -T0 > ${DEPLOY_DIR_IMAGE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz >>> + bbnote "SDK rootfs available in ${DEPLOY_DIR_IMAGE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz" >>> +} >>> + >>> +sdk_container_images() { >>> + local cmd="/bin/dash" >>> + local empty_tag="empty" >>> + local full_tag="latest" >>> + local oci_img_dir="${WORKDIR}/oci-image" >>> + local sdk_container_formats="$1" >>> + >>> + # prepare OCI container image skeleton >>> + sudo umoci init --layout "${oci_img_dir}" >>> + sudo umoci new --image "${oci_img_dir}:${empty_tag}" >>> + sudo umoci config --image "${oci_img_dir}:${empty_tag}" \ >>> + --config.cmd="${cmd}" >>> + sudo umoci unpack --image "${oci_img_dir}:${empty_tag}" \ >>> + "${oci_img_dir}_unpacked" >>> + >>> + # add SDK root filesystem as the flesh of the skeleton >>> + sudo cp -a "${SDKCHROOT_DIR}"/* "${oci_img_dir}_unpacked/rootfs/" >>> + >>> + # pack container image >>> + sudo umoci repack --image "${oci_img_dir}:${full_tag}" \ >>> + "${oci_img_dir}_unpacked" >>> + sudo umoci remove --image "${oci_img_dir}:${empty_tag}" >>> + sudo rm -rf "${oci_img_dir}_unpacked" >>> + >>> + # no root needed anymore >>> + sudo chown --recursive $(id -u):$(id -g) "${oci_img_dir}" >>> + >>> + # convert the OCI container image to the desired format >>> + sdk_id="sdk-${DISTRO}-${DISTRO_ARCH}" >>> + image_name="isar-${sdk_id}" >>> + image_archive="${DEPLOY_DIR_IMAGE}/${sdk_id}-${sdk_format}.tar" >>> + for sdk_format in ${sdk_container_formats} ; do >>> + case "${sdk_format}" in >>> + "docker-archive" | "oci-archive") >>> + if [ "${sdk_format}" = "oci-archive" ] ; then >>> + target="${sdk_format}:${image_archive}:latest" >>> + else >>> + target="${sdk_format}:${image_archive}:${image_name}:latest" >>> + fi >>> + skopeo --insecure-policy copy \ >>> + "oci:${oci_img_dir}:${full_tag}" "${target}" >>> + xz -T0 "${image_archive}" >>> + bbnote "Containerized SDK available in ${image_archive}.xz" >>> + ;; >>> + "oci") >>> + tar --create --xz --directory "${oci_img_dir}" \ >>> + --file "${image_archive}.xz" . >>> + bbnote "Containerized SDK available in ${image_archive}.xz" >>> + ;; >>> + "docker-daemon" | "containers-storage") >>> + skopeo --insecure-policy copy \ >>> + "oci:${oci_img_dir}:${full_tag}" \ >>> + "${sdk_format}:${image_name}:latest" >>> + bbnote "Containerized SDK available in ${sdk_format} as '${image_name}:latest'" >>> + ;; >>> + esac >>> + done >>> +} >>> >>> do_populate_sdk[stamp-extra-info] = "${DISTRO}-${MACHINE}" >>> do_populate_sdk[depends] = "sdkchroot:do_build" >>> -do_populate_sdk[vardeps] += "SDK_INCLUDE_ISAR_APT" >>> +do_populate_sdk[vardeps] += "SDK_INCLUDE_ISAR_APT SDK_FORMATS" >>> do_populate_sdk() { >>> if [ "${SDK_INCLUDE_ISAR_APT}" = "1" ]; then >>> # Copy isar-apt with deployed Isar packages >>> @@ -48,12 +119,31 @@ do_populate_sdk() { >>> done >>> done >>> >>> - # Copy mount_chroot.sh for convenience >>> - sudo cp ${SCRIPTSDIR}/mount_chroot.sh ${SDKCHROOT_DIR} >>> + # separate SDK formats: TAR and container formats >>> + container_formats="" >>> + for sdk_format in ${SDK_FORMATS} ; do >>> + case ${sdk_format} in >>> + "tar-xz") >>> + sdk_tar_xz >>> + ;; >>> + "docker-archive" | "oci" | "oci-archive") >>> + container_formats="${container_formats} ${sdk_format}" >>> + ;; >>> + "docker-daemon" | "containers-storage") >>> + if [ -f /.dockerenv ] || [ -f /run/.containerenv ] ; then >>> + die "Adding the SDK container image to a container runtime (${sdk_format}) not supported if running from a container (e.g. 'kas-container')" >>> + fi >>> + ;; >>> + *) >>> + die "unsupported SDK format specified: ${sdk_format}" >>> + ;; >>> + esac >>> + done >>> >>> - # Create SDK archive >>> - cd -P ${SDKCHROOT_DIR}/.. >>> - sudo tar --transform="s|^rootfs|sdk-${DISTRO}-${DISTRO_ARCH}|" \ >>> - -c rootfs | xz -T0 > ${DEPLOY_DIR_IMAGE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz >>> + # generate the SDK in all the desired container formats >>> + if [ -n "${container_formats}" ] ; then >>> + bbnote "Generating SDK container in${container_formats} format" >>> + sdk_container_images "${container_formats}" >>> + fi >>> } >>> addtask populate_sdk after do_rootfs >>> >> How much of this would be reusable of generating a container from a >> target rootfs? We should avoid shuffling code around if we can already >> line things up nicely while introducing it. > > With that reuse in mind I can refactor the code to have a class that can be reused for both SDK and target rootfs. It's not a big deal. > > Is it somehow related to the huge "some image classes" thread that is active right now? > Not directly related, but container images would fall into that same category. Jan -- Siemens AG, T RDA IOT Corporate Competence Center Embedded Linux