From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6927266035414335488 X-Received: by 2002:a17:906:4cd6:: with SMTP id q22mr4017183ejt.469.1615393090167; Wed, 10 Mar 2021 08:18:10 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:906:c192:: with SMTP id g18ls1422279ejz.10.gmail; Wed, 10 Mar 2021 08:18:09 -0800 (PST) X-Google-Smtp-Source: ABdhPJyjOuwCdIbq9LpJqjxsWJaDaXedGl2V44iBPaX4ek/oYcxXOPeGe5ajjJIv79ViOxw4BgLj X-Received: by 2002:a17:906:8043:: with SMTP id x3mr4392980ejw.149.1615393089206; Wed, 10 Mar 2021 08:18:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1615393089; cv=none; d=google.com; s=arc-20160816; b=TGLk6rF0ZKqElLOdAB6LF5L+qJrJ94QJGl/6l48flDv4bxLEvO/8th3/O2CdI7hEjo HNca+E3QWQX3YNmRkhxxAYXEca4/ioDjX6aw1WdOJWU1B72dUIV52wirSvUG422iPvBS HZf1K2fAao3VwMjCgtQzAuAljU6TzDhWbA2wSDa2nksQ6pmGXXRvJBKwUfxHJcQ/08vx o1J6mfhzi2UrahU00UzBw1tLN7aiD9mLEo6wwGIEfrKM4XNwMfofIzoD1IWllLEKaX3R HwO8wz6m35cl23FIQpiSGx/0Fvgty12V5gJekP6JbZuH5KYm+RL07fC4Y+1MUMqoj7D/ xZfg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-language:content-transfer-encoding:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject; bh=DsgKoPqxwd23BbB7aD1EbJ0iZRCCWiOZ69LTWFL4cIY=; b=jlB0BEaFv0uBJQJRFgj0N3ghsm6AxoiXQDHuVPUQCt85/UlmevxxLlqmxeEOdhoiMc rmBkbeSJ3HcRzLTy0rb9vP3e2FF/qKPpQGXafx0GtaLzYClxZVLBr/3GEUlmYFUPcm3T CixeBnVSFFdfAvu/4zI9p0F7ouOOlQsmDwnJ0yPKVkQwjME9S2LyC9M2nzKVCD30vriL iKPvkhJJW8dcYG3xb6U/hqKX7qUkb8je85WmFzv1uDr4Kjdh789qSlF8Fz9R5eHz2A4u M2FaMI+XHMuje8AI6D6xMi/tpV0Ju75El5m632jR0dE1wo64bmmXA9D/F4deEzGl3VxL J6/g== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id t7si790159edr.0.2021.03.10.08.18.09 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 10 Mar 2021 08:18:09 -0800 (PST) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from [192.168.67.164] (mm-100-87-121-178.mgts.dynamic.pppoe.byfly.by [178.121.87.100] (may be forged)) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8) with ESMTPSA id 12AGI69d003694 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 10 Mar 2021 17:18:07 +0100 Subject: Re: [PATCH v6 0/5] Add support for containerized root filesystems To: Silvano Cirujano Cuesta , isar-users@googlegroups.com References: <20210309205239.652677-1-silvano.cirujano-cuesta@siemens.com> From: Anton Mikanovich Message-ID: Date: Wed, 10 Mar 2021 19:18:01 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.7.1 MIME-Version: 1.0 In-Reply-To: <20210309205239.652677-1-silvano.cirujano-cuesta@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 8bit Content-Language: en-US X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: ruFhqbLk0Pzo 09.03.2021 23:52, Silvano Cirujano Cuesta wrote: > This patch series provides support for containerized root filesystems, > for both target images and SDKs. > > For containerized target images the new image type `container-img` has > been added. > > For containerized SDKs the task `populate_sdk` has been extended. > > Containerized root filesystems are easy to distribute and run, enabling > this way following scenarios: > - Use ISAR to build container images meant to be run only in containers. > - Use the same ISAR configuration to build images for containers, VMs > and bare-metal. > - Easy SDK distribution and "installation". > - Quickly testing certain applications in the workstation using the > target root filesystem. > > In order to build containerized target root filesystems `IMAGE_TYPE` has > to be `container-img`, additionally the container image format can be > selected with the variable `CONTAINER_FORMATS`. The default format is > `docker-archive`. > > In order to build containerized SDKs the variable `SDK_FORMAT` has to > provide any of the supported container formats (e.g. `docker-archive`). > The default format is the legacy non-containerized: `tar_xz`. > > It also provides a sample machine, multiconfigs and ci-testing. > > More information about its usage is documented in the file > docs/user_manual.md. > > A PoC/demo of this functionality (only the SDK part) has been created > based on the project https://github.com/siemens/meta-iot2050. > Jan Kiszka already tested and liked it! => > https://github.com/siemens/meta-iot2050/issues/86#issuecomment-768907845 > > Successful builds of both containerized target and SDK are available on > the same PoC project: > - https://github.com/Silvanoc/meta-iot2050/actions/runs/558311580 > - https://github.com/Silvanoc/meta-iot2050/actions/runs/558311581 > and also the resulting images: > - https://github.com/users/Silvanoc/packages/container/package/meta-iot2050%2Fiot2050-debian-arm64 > - https://github.com/users/Silvanoc/packages/container/package/meta-iot2050%2Fiot2050-debian-sdk-arm64 > > In order to get a feeling about its usage (you need Docker or Podman), > follow these simple copy&paste instructions: > https://github.com/Silvanoc/meta-iot2050/blob/master/kas/BUILDING-SDK-CONTAINER.md#running-the-sdk > Build instructions are available in the upper part of that document. > > Two new dependencies (umoci and skopeo -backporting from bullseye to > buster works easily) are required to create containerized root > filesystems (as specified in the documentation). > > Typical container image management actions (e.g. push an image to a > container image regitry) are out of scope. Available tools (Docker, > Skopeo, Buildah, Podman,...) should be used for these actions. > > A patch will follow this one to get the dependencies into the container > images being provided by the project > https://github.com/siemens/kas (for `kas-container`, for example). > > Silvano Cirujano Cuesta (5): > classes: add root filesystem containerizing class > classes: add new image type 'container-img' > sdk: add support for containerized sdk > docs: document creation of container images > ci: add container image testing configurations > > doc/user_manual.md | 127 ++++++++++++++++++ > meta-isar/conf/machine/container.conf | 5 + > .../conf/multiconfig/container-bullseye.conf | 4 + > .../conf/multiconfig/container-buster.conf | 4 + > .../conf/multiconfig/container-focal.conf | 4 + > .../conf/multiconfig/container-stretch.conf | 4 + > meta/classes/container-img.bbclass | 18 +++ > .../classes/image-container-extension.bbclass | 81 +++++++++++ > meta/classes/image-sdk-extension.bbclass | 42 +++++- > meta/classes/image.bbclass | 1 + > scripts/ci_build.sh | 11 +- > 11 files changed, 293 insertions(+), 8 deletions(-) > create mode 100644 meta-isar/conf/machine/container.conf > create mode 100644 meta-isar/conf/multiconfig/container-bullseye.conf > create mode 100644 meta-isar/conf/multiconfig/container-buster.conf > create mode 100644 meta-isar/conf/multiconfig/container-focal.conf > create mode 100644 meta-isar/conf/multiconfig/container-stretch.conf > create mode 100644 meta/classes/container-img.bbclass > create mode 100644 meta/classes/image-container-extension.bbclass > Thanks for adding CI cases, but Jenkins failed during do_populate_sdk: 14:17:17 + SDK_FORMAT=docker-archive 14:17:17 + BB_ENV_WHITELIST=' SDK_FORMAT' 14:17:17 + bitbake -c do_populate_sdk mc:qemuarm-stretch:isar-image-base ... 14:17:18 ERROR: Unable to parse /workspace/build/isar_am_devel_fast/92/meta-isar/recipes-app/libhello/libhello.bb 14:17:18 Traceback (most recent call last): 14:17:18   File "/workspace/build/isar_am_devel_fast/92/bitbake/lib/bb/parse/ast.py", line 382, in multi_finalize(fn='/workspace/build/isar_am_devel_fast/92/meta-isar/recipes-app/libhello/libhello.bb', d=): 14:17:18          try: 14:17:18     >        finalize(fn, d) 14:17:18          except bb.parse.SkipRecipe as e: 14:17:18   File "/workspace/build/isar_am_devel_fast/92/bitbake/lib/bb/parse/ast.py", line 347, in finalize(fn='/workspace/build/isar_am_devel_fast/92/meta-isar/recipes-app/libhello/libhello.bb', d=, variant=None): 14:17:18 14:17:18     >        bb.parse.siggen.finalise(fn, d, variant) 14:17:18 14:17:18   File "/workspace/build/isar_am_devel_fast/92/bitbake/lib/bb/siggen.py", line 182, in SignatureGeneratorBasicHash.finalise(fn='/workspace/build/isar_am_devel_fast/92/meta-isar/recipes-app/libhello/libhello.bb', d=, variant=None): 14:17:18              try: 14:17:18     >            taskdeps = self._build_data(fn, d) 14:17:18              except bb.parse.SkipRecipe: 14:17:18   File "/workspace/build/isar_am_devel_fast/92/bitbake/lib/bb/siggen.py", line 150, in SignatureGeneratorBasicHash._build_data(fn='/workspace/build/isar_am_devel_fast/92/meta-isar/recipes-app/libhello/libhello.bb', d=): 14:17:18              ignore_mismatch = ((d.getVar("BB_HASH_IGNORE_MISMATCH") or '') == '1') 14:17:18     >        tasklist, gendeps, lookupcache = bb.data.generate_dependencies(d, self.basewhitelist) 14:17:18 14:17:18   File "/workspace/build/isar_am_devel_fast/92/bitbake/lib/bb/data.py", line 371, in generate_dependencies(d=, whitelist={'SHELL', 'DEPLOY_DIR', 'PRSERV_DUMPFILE', 'SDKPKGSUFFIX', 'FILE_DIRNAME', 'TERM', 'COREBASE', 'LICENSE_PATH', 'STAGING_DIR_HOST', 'SSTATE_PKGARCH', 'PKGDATA_DIR', 'THISDIR', 'USER', 'EXTERNAL_TOOLCHAIN', 'PWD', 'DL_DIR', 'LOGNAME', 'PATH', 'BB_LIMITEDDEPS', 'CCACHE', 'BB_TASKHASH', 'CCACHE_DIR', 'PARALLEL_MAKE', 'BBSERVER', 'PRSERV_LOCKDOWN', 'BBPATH', 'BB_WORKERCONTEXT', 'PRSERV_HOST', 'WORKDIR', 'FILE', 'CCACHE_NOHASHDIR', 'PRSERV_DUMPDIR', 'FILESEXTRAPATHS', 'HOME', 'BUILD_ARCH', 'STAMPCLEAN', 'SSTATE_DIR', 'TMPDIR', 'FILESPATH', 'STAGING_DIR_TARGET'}): 14:17:18          keys = set(key for key in d if not key.startswith("__")) 14:17:18     >    shelldeps = set(key for key in d.getVar("__exportlist", False) if d.getVarFlag(key, "export", False) and not d.getVarFlag(key, "unexport", False)) 14:17:18          varflagsexcl = d.getVar('BB_SIGNATURE_EXCLUDE_FLAGS') 14:17:18 TypeError: 'NoneType' object is not iterable Full log: http://ci.isar-build.org:8080/job/isar_am_devel_fast/92/consoleFull -- Anton Mikanovich Promwad Ltd. External service provider of ilbers GmbH Maria-Merian-Str. 8 85521 Ottobrunn, Germany +49 (89) 122 67 24-0 Commercial register Munich, HRB 214197 General Manager: Baurzhan Ismagulov