Re: [PATCH v6 0/5] Add support for containerized root filesystems

[Anton Mikanovich <amikan@ilbers.de>]

09.03.2021 23:52, Silvano Cirujano Cuesta wrote:
> This patch series provides support for containerized root filesystems,
> for both target images and SDKs.
>
> For containerized target images the new image type `container-img` has
> been added.
>
> For containerized SDKs the task `populate_sdk` has been extended.
>
> Containerized root filesystems are easy to distribute and run, enabling
> this way following scenarios:
>   - Use ISAR to build container images meant to be run only in containers.
>   - Use the same ISAR configuration to build images for containers, VMs
>     and bare-metal.
>   - Easy SDK distribution and "installation".
>   - Quickly testing certain applications in the workstation using the
>     target root filesystem.
>
> In order to build containerized target root filesystems `IMAGE_TYPE` has
> to be `container-img`, additionally the container image format can be
> selected with the variable `CONTAINER_FORMATS`. The default format is
> `docker-archive`.
>
> In order to build containerized SDKs the variable `SDK_FORMAT` has to
> provide any of the supported container formats (e.g. `docker-archive`).
> The default format is the legacy non-containerized: `tar_xz`.
>
> It also provides a sample machine, multiconfigs and ci-testing.
>
> More information about its usage is documented in the file
> docs/user_manual.md.
>
> A PoC/demo of this functionality (only the SDK part) has been created
> based on the project https://github.com/siemens/meta-iot2050.
> Jan Kiszka already tested and liked it! =>
> https://github.com/siemens/meta-iot2050/issues/86#issuecomment-768907845
>
> Successful builds of both containerized target and SDK are available on
> the same PoC project:
>   - https://github.com/Silvanoc/meta-iot2050/actions/runs/558311580
>   - https://github.com/Silvanoc/meta-iot2050/actions/runs/558311581
> and also the resulting images:
>   - https://github.com/users/Silvanoc/packages/container/package/meta-iot2050%2Fiot2050-debian-arm64
>   - https://github.com/users/Silvanoc/packages/container/package/meta-iot2050%2Fiot2050-debian-sdk-arm64
>
> In order to get a feeling about its usage (you need Docker or Podman),
> follow these simple copy&paste instructions:
> https://github.com/Silvanoc/meta-iot2050/blob/master/kas/BUILDING-SDK-CONTAINER.md#running-the-sdk
> Build instructions are available in the upper part of that document.
>
> Two new dependencies (umoci and skopeo -backporting from bullseye to
> buster works easily) are required to create containerized root
> filesystems (as specified in the documentation).
>
> Typical container image management actions (e.g. push an image to a
> container image regitry) are out of scope. Available tools (Docker,
> Skopeo, Buildah, Podman,...) should be used for these actions.
>
> A patch will follow this one to get the dependencies into the container
> images being provided by the project
> https://github.com/siemens/kas (for `kas-container`, for example).
>
> Silvano Cirujano Cuesta (5):
>    classes: add root filesystem containerizing class
>    classes: add new image type 'container-img'
>    sdk: add support for containerized sdk
>    docs: document creation of container images
>    ci: add container image testing configurations
>
>   doc/user_manual.md                            | 127 ++++++++++++++++++
>   meta-isar/conf/machine/container.conf         |   5 +
>   .../conf/multiconfig/container-bullseye.conf  |   4 +
>   .../conf/multiconfig/container-buster.conf    |   4 +
>   .../conf/multiconfig/container-focal.conf     |   4 +
>   .../conf/multiconfig/container-stretch.conf   |   4 +
>   meta/classes/container-img.bbclass            |  18 +++
>   .../classes/image-container-extension.bbclass |  81 +++++++++++
>   meta/classes/image-sdk-extension.bbclass      |  42 +++++-
>   meta/classes/image.bbclass                    |   1 +
>   scripts/ci_build.sh                           |  11 +-
>   11 files changed, 293 insertions(+), 8 deletions(-)
>   create mode 100644 meta-isar/conf/machine/container.conf
>   create mode 100644 meta-isar/conf/multiconfig/container-bullseye.conf
>   create mode 100644 meta-isar/conf/multiconfig/container-buster.conf
>   create mode 100644 meta-isar/conf/multiconfig/container-focal.conf
>   create mode 100644 meta-isar/conf/multiconfig/container-stretch.conf
>   create mode 100644 meta/classes/container-img.bbclass
>   create mode 100644 meta/classes/image-container-extension.bbclass
>
Thanks for adding CI cases, but Jenkins failed during do_populate_sdk:

14:17:17 + SDK_FORMAT=docker-archive
14:17:17 + BB_ENV_WHITELIST=' SDK_FORMAT'
14:17:17 + bitbake -c do_populate_sdk mc:qemuarm-stretch:isar-image-base
...
14:17:18 ERROR: Unable to parse 
/workspace/build/isar_am_devel_fast/92/meta-isar/recipes-app/libhello/libhello.bb
14:17:18 Traceback (most recent call last):
14:17:18   File 
"/workspace/build/isar_am_devel_fast/92/bitbake/lib/bb/parse/ast.py", 
line 382, in 
multi_finalize(fn='/workspace/build/isar_am_devel_fast/92/meta-isar/recipes-app/libhello/libhello.bb', 
d=<bb.data_smart.DataSmart object at 0x7f12917e1c50>):
14:17:18          try:
14:17:18     >        finalize(fn, d)
14:17:18          except bb.parse.SkipRecipe as e:
14:17:18   File 
"/workspace/build/isar_am_devel_fast/92/bitbake/lib/bb/parse/ast.py", 
line 347, in 
finalize(fn='/workspace/build/isar_am_devel_fast/92/meta-isar/recipes-app/libhello/libhello.bb', 
d=<bb.data_smart.DataSmart object at 0x7f12917e1c50>, variant=None):
14:17:18
14:17:18     >        bb.parse.siggen.finalise(fn, d, variant)
14:17:18
14:17:18   File 
"/workspace/build/isar_am_devel_fast/92/bitbake/lib/bb/siggen.py", line 
182, in 
SignatureGeneratorBasicHash.finalise(fn='/workspace/build/isar_am_devel_fast/92/meta-isar/recipes-app/libhello/libhello.bb', 
d=<bb.data_smart.DataSmart object at 0x7f12917e1c50>, variant=None):
14:17:18              try:
14:17:18     >            taskdeps = self._build_data(fn, d)
14:17:18              except bb.parse.SkipRecipe:
14:17:18   File 
"/workspace/build/isar_am_devel_fast/92/bitbake/lib/bb/siggen.py", line 
150, in 
SignatureGeneratorBasicHash._build_data(fn='/workspace/build/isar_am_devel_fast/92/meta-isar/recipes-app/libhello/libhello.bb', 
d=<bb.data_smart.DataSmart object at 0x7f12917e1c50>):
14:17:18              ignore_mismatch = 
((d.getVar("BB_HASH_IGNORE_MISMATCH") or '') == '1')
14:17:18     >        tasklist, gendeps, lookupcache = 
bb.data.generate_dependencies(d, self.basewhitelist)
14:17:18
14:17:18   File 
"/workspace/build/isar_am_devel_fast/92/bitbake/lib/bb/data.py", line 
371, in generate_dependencies(d=<bb.data_smart.DataSmart object at 
0x7f12917e1c50>, whitelist={'SHELL', 'DEPLOY_DIR', 'PRSERV_DUMPFILE', 
'SDKPKGSUFFIX', 'FILE_DIRNAME', 'TERM', 'COREBASE', 'LICENSE_PATH', 
'STAGING_DIR_HOST', 'SSTATE_PKGARCH', 'PKGDATA_DIR', 'THISDIR', 'USER', 
'EXTERNAL_TOOLCHAIN', 'PWD', 'DL_DIR', 'LOGNAME', 'PATH', 
'BB_LIMITEDDEPS', 'CCACHE', 'BB_TASKHASH', 'CCACHE_DIR', 
'PARALLEL_MAKE', 'BBSERVER', 'PRSERV_LOCKDOWN', 'BBPATH', 
'BB_WORKERCONTEXT', 'PRSERV_HOST', 'WORKDIR', 'FILE', 
'CCACHE_NOHASHDIR', 'PRSERV_DUMPDIR', 'FILESEXTRAPATHS', 'HOME', 
'BUILD_ARCH', 'STAMPCLEAN', 'SSTATE_DIR', 'TMPDIR', 'FILESPATH', 
'STAGING_DIR_TARGET'}):
14:17:18          keys = set(key for key in d if not key.startswith("__"))
14:17:18     >    shelldeps = set(key for key in 
d.getVar("__exportlist", False) if d.getVarFlag(key, "export", False) 
and not d.getVarFlag(key, "unexport", False))
14:17:18          varflagsexcl = d.getVar('BB_SIGNATURE_EXCLUDE_FLAGS')
14:17:18 TypeError: 'NoneType' object is not iterable

Full log: 
http://ci.isar-build.org:8080/job/isar_am_devel_fast/92/consoleFull

-- 
Anton Mikanovich
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov