From: Henning Schild <henning.schild@siemens.com>
To: "[ext] Silvano Cirujano Cuesta" <silvano.cirujano-cuesta@siemens.com>
Cc: Anton Mikanovich <amikan@ilbers.de>, isar-users@googlegroups.com
Subject: Re: [PATCH v6 0/5] Add support for containerized root filesystems
Date: Wed, 10 Mar 2021 18:25:14 +0100 [thread overview]
Message-ID: <20210310182514.25b0c08b@md1za8fc.ad001.siemens.net> (raw)
In-Reply-To: <3d304c0a-a7e7-99c7-fdf0-4946a6a59a7b@siemens.com>
Am Wed, 10 Mar 2021 18:18:23 +0100
schrieb "[ext] Silvano Cirujano Cuesta"
<silvano.cirujano-cuesta@siemens.com>:
> On 10/03/2021 17:18, Anton Mikanovich wrote:
> > 09.03.2021 23:52, Silvano Cirujano Cuesta wrote:
> >> This patch series provides support for containerized root
> >> filesystems, for both target images and SDKs.
> >>
> >> For containerized target images the new image type `container-img`
> >> has been added.
> >>
> >> For containerized SDKs the task `populate_sdk` has been extended.
> >>
> >> Containerized root filesystems are easy to distribute and run,
> >> enabling this way following scenarios:
> >> - Use ISAR to build container images meant to be run only in
> >> containers.
> >> - Use the same ISAR configuration to build images for
> >> containers, VMs and bare-metal.
> >> - Easy SDK distribution and "installation".
> >> - Quickly testing certain applications in the workstation using
> >> the target root filesystem.
> >>
> >> In order to build containerized target root filesystems
> >> `IMAGE_TYPE` has to be `container-img`, additionally the container
> >> image format can be selected with the variable
> >> `CONTAINER_FORMATS`. The default format is `docker-archive`.
> >>
> >> In order to build containerized SDKs the variable `SDK_FORMAT` has
> >> to provide any of the supported container formats (e.g.
> >> `docker-archive`). The default format is the legacy
> >> non-containerized: `tar_xz`.
> >>
> >> It also provides a sample machine, multiconfigs and ci-testing.
> >>
> >> More information about its usage is documented in the file
> >> docs/user_manual.md.
> >>
> >> A PoC/demo of this functionality (only the SDK part) has been
> >> created based on the project
> >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsiemens%2Fmeta-iot2050&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C70ab5dc719fe445169a008d8e3e01c27%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637509898980089588%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2FB%2BNBKhRCkVNy%2FGEbAiFgEUZBpy0VmhkWdpvNq2wokM%3D&reserved=0.
> >> Jan Kiszka already tested and liked it! =>
> >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsiemens%2Fmeta-iot2050%2Fissues%2F86%23issuecomment-768907845&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C70ab5dc719fe445169a008d8e3e01c27%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637509898980089588%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=Os3cY15D2myT5sdPINARBh63iwb3GpE5IlupRGnA0v0%3D&reserved=0
> >>
> >> Successful builds of both containerized target and SDK are
> >> available on the same PoC project:
> >> -
> >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSilvanoc%2Fmeta-iot2050%2Factions%2Fruns%2F558311580&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C70ab5dc719fe445169a008d8e3e01c27%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637509898980089588%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=N3DUdGxhNOWlvbatGxopSWPWsMXsXdvsKmIsOh1KtyU%3D&reserved=0
> >> -
> >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSilvanoc%2Fmeta-iot2050%2Factions%2Fruns%2F558311581&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C70ab5dc719fe445169a008d8e3e01c27%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637509898980089588%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=cOkongJ%2BtbrEvaquB241NbaK1nQHHiY%2FrgbYHPFAjeo%3D&reserved=0
> >> and also the resulting images:
> >> -
> >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fusers%2FSilvanoc%2Fpackages%2Fcontainer%2Fpackage%2Fmeta-iot2050%252Fiot2050-debian-arm64&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C70ab5dc719fe445169a008d8e3e01c27%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637509898980089588%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=UegaXdy58CUxX8%2FsDnG8cy3V9uy9IUyh8yA00U%2FSaR4%3D&reserved=0
> >> -
> >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fusers%2FSilvanoc%2Fpackages%2Fcontainer%2Fpackage%2Fmeta-iot2050%252Fiot2050-debian-sdk-arm64&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C70ab5dc719fe445169a008d8e3e01c27%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637509898980089588%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=wss3OIVsS2ulaQ4Jr7ria0wMna6Y0U8Ahp9M3XQU5PM%3D&reserved=0
> >>
> >> In order to get a feeling about its usage (you need Docker or
> >> Podman), follow these simple copy&paste instructions:
> >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2FSilvanoc%2Fmeta-iot2050%2Fblob%2Fmaster%2Fkas%2FBUILDING-SDK-CONTAINER.md%23running-the-sdk&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C70ab5dc719fe445169a008d8e3e01c27%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637509898980089588%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=%2FiIv247NEqKV%2F2osF6hL5S4QiA6T8yqd5TQpi3EbuGQ%3D&reserved=0
> >> Build instructions are available in the upper part of that
> >> document.
> >>
> >> Two new dependencies (umoci and skopeo -backporting from bullseye
> >> to buster works easily) are required to create containerized root
> >> filesystems (as specified in the documentation).
> >>
> >> Typical container image management actions (e.g. push an image to a
> >> container image regitry) are out of scope. Available tools (Docker,
> >> Skopeo, Buildah, Podman,...) should be used for these actions.
> >>
> >> A patch will follow this one to get the dependencies into the
> >> container images being provided by the project
> >> https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fsiemens%2Fkas&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C70ab5dc719fe445169a008d8e3e01c27%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637509898980099579%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=eozG3LQR9pRGIcNiz9a0DckXhkBu8pcas9c%2F%2BWon9mU%3D&reserved=0
> >> (for `kas-container`, for example).
> >>
> >> Silvano Cirujano Cuesta (5):
> >> classes: add root filesystem containerizing class
> >> classes: add new image type 'container-img'
> >> sdk: add support for containerized sdk
> >> docs: document creation of container images
> >> ci: add container image testing configurations
> >>
> >> doc/user_manual.md | 127
> >> ++++++++++++++++++ meta-isar/conf/machine/container.conf |
> >> 5 + .../conf/multiconfig/container-bullseye.conf | 4 +
> >> .../conf/multiconfig/container-buster.conf | 4 +
> >> .../conf/multiconfig/container-focal.conf | 4 +
> >> .../conf/multiconfig/container-stretch.conf | 4 +
> >> meta/classes/container-img.bbclass | 18 +++
> >> .../classes/image-container-extension.bbclass | 81 +++++++++++
> >> meta/classes/image-sdk-extension.bbclass | 42 +++++-
> >> meta/classes/image.bbclass | 1 +
> >> scripts/ci_build.sh | 11 +-
> >> 11 files changed, 293 insertions(+), 8 deletions(-)
> >> create mode 100644 meta-isar/conf/machine/container.conf
> >> create mode 100644
> >> meta-isar/conf/multiconfig/container-bullseye.conf create mode
> >> 100644 meta-isar/conf/multiconfig/container-buster.conf create
> >> mode 100644 meta-isar/conf/multiconfig/container-focal.conf create
> >> mode 100644 meta-isar/conf/multiconfig/container-stretch.conf
> >> create mode 100644 meta/classes/container-img.bbclass create mode
> >> 100644 meta/classes/image-container-extension.bbclass
> > Thanks for adding CI cases, but Jenkins failed during
> > do_populate_sdk:
>
> Do you have the documented dependencies (umoci and skopeo backported
> from bullseye) in the CI system? I've tested the commands manually on
> my system using kas-container and they worked, but your setup is
> slightly different.
AFAIK the jenkins runner is debian buster running "shell" jobs. New
deps will not work here.
I did not review the changes, but hope those new deps have been added
to the docs.
Henning
> >
> > 14:17:17 + SDK_FORMAT=docker-archive
> > 14:17:17 + BB_ENV_WHITELIST=' SDK_FORMAT'
> > 14:17:17 + bitbake -c do_populate_sdk
> > mc:qemuarm-stretch:isar-image-base ...
> > 14:17:18 ERROR: Unable to parse
> > /workspace/build/isar_am_devel_fast/92/meta-isar/recipes-app/libhello/libhello.bb
> > 14:17:18 Traceback (most recent call last): 14:17:18 File
> > "/workspace/build/isar_am_devel_fast/92/bitbake/lib/bb/parse/ast.py",
> > line 382, in
> > multi_finalize(fn='/workspace/build/isar_am_devel_fast/92/meta-isar/recipes-app/libhello/libhello.bb',
> > d=<bb.data_smart.DataSmart object at 0x7f12917e1c50>): 14:17:18
> > try: 14:17:18 > finalize(fn, d) 14:17:18
> > except bb.parse.SkipRecipe as e: 14:17:18 File
> > "/workspace/build/isar_am_devel_fast/92/bitbake/lib/bb/parse/ast.py",
> > line 347, in
> > finalize(fn='/workspace/build/isar_am_devel_fast/92/meta-isar/recipes-app/libhello/libhello.bb',
> > d=<bb.data_smart.DataSmart object at 0x7f12917e1c50>,
> > variant=None): 14:17:18 14:17:18 >
> > bb.parse.siggen.finalise(fn, d, variant) 14:17:18 14:17:18 File
> > "/workspace/build/isar_am_devel_fast/92/bitbake/lib/bb/siggen.py",
> > line 182, in
> > SignatureGeneratorBasicHash.finalise(fn='/workspace/build/isar_am_devel_fast/92/meta-isar/recipes-app/libhello/libhello.bb',
> > d=<bb.data_smart.DataSmart object at 0x7f12917e1c50>,
> > variant=None): 14:17:18 try: 14:17:18 >
> > taskdeps = self._build_data(fn, d) 14:17:18 except
> > bb.parse.SkipRecipe: 14:17:18 File
> > "/workspace/build/isar_am_devel_fast/92/bitbake/lib/bb/siggen.py",
> > line 150, in
> > SignatureGeneratorBasicHash._build_data(fn='/workspace/build/isar_am_devel_fast/92/meta-isar/recipes-app/libhello/libhello.bb',
> > d=<bb.data_smart.DataSmart object at 0x7f12917e1c50>): 14:17:18
> > ignore_mismatch = ((d.getVar("BB_HASH_IGNORE_MISMATCH") or
> > '') == '1') 14:17:18 > tasklist, gendeps, lookupcache =
> > bb.data.generate_dependencies(d, self.basewhitelist) 14:17:18
> > 14:17:18 File
> > "/workspace/build/isar_am_devel_fast/92/bitbake/lib/bb/data.py",
> > line 371, in generate_dependencies(d=<bb.data_smart.DataSmart
> > object at 0x7f12917e1c50>, whitelist={'SHELL', 'DEPLOY_DIR',
> > 'PRSERV_DUMPFILE', 'SDKPKGSUFFIX', 'FILE_DIRNAME', 'TERM',
> > 'COREBASE', 'LICENSE_PATH', 'STAGING_DIR_HOST', 'SSTATE_PKGARCH',
> > 'PKGDATA_DIR', 'THISDIR', 'USER', 'EXTERNAL_TOOLCHAIN', 'PWD',
> > 'DL_DIR', 'LOGNAME', 'PATH', 'BB_LIMITEDDEPS', 'CCACHE',
> > 'BB_TASKHASH', 'CCACHE_DIR', 'PARALLEL_MAKE', 'BBSERVER',
> > 'PRSERV_LOCKDOWN', 'BBPATH', 'BB_WORKERCONTEXT', 'PRSERV_HOST',
> > 'WORKDIR', 'FILE', 'CCACHE_NOHASHDIR', 'PRSERV_DUMPDIR',
> > 'FILESEXTRAPATHS', 'HOME', 'BUILD_ARCH', 'STAMPCLEAN',
> > 'SSTATE_DIR', 'TMPDIR', 'FILESPATH', 'STAGING_DIR_TARGET'}):
> > 14:17:18 keys = set(key for key in d if not
> > key.startswith("__")) 14:17:18 > shelldeps = set(key for key
> > in d.getVar("__exportlist", False) if d.getVarFlag(key, "export",
> > False) and not d.getVarFlag(key, "unexport", False)) 14:17:18
> > varflagsexcl = d.getVar('BB_SIGNATURE_EXCLUDE_FLAGS') 14:17:18
> > TypeError: 'NoneType' object is not iterable
> >
> > Full log:
> > https://eur01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fci.isar-build.org%3A8080%2Fjob%2Fisar_am_devel_fast%2F92%2FconsoleFull&data=04%7C01%7Csilvano.cirujano-cuesta%40siemens.com%7C70ab5dc719fe445169a008d8e3e01c27%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637509898980099579%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=SS%2FYypaMBwrC9dvwJCeIrW9EjPqVfF2RXFghcfAn4Lc%3D&reserved=0
>
next prev parent reply other threads:[~2021-03-10 17:25 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-09 20:52 Silvano Cirujano Cuesta
2021-03-09 20:52 ` [PATCH v6 1/5] classes: add root filesystem containerizing class Silvano Cirujano Cuesta
2021-03-09 20:52 ` [PATCH v6 2/5] classes: add new image type 'container-img' Silvano Cirujano Cuesta
2021-03-09 20:52 ` [PATCH v6 3/5] sdk: add support for containerized sdk Silvano Cirujano Cuesta
2021-03-09 20:52 ` [PATCH v6 4/5] docs: document creation of container images Silvano Cirujano Cuesta
2021-03-09 20:52 ` [PATCH v6 5/5] ci: add container image testing configurations Silvano Cirujano Cuesta
2021-03-10 16:18 ` [PATCH v6 0/5] Add support for containerized root filesystems Anton Mikanovich
2021-03-10 17:18 ` Silvano Cirujano Cuesta
2021-03-10 17:25 ` Henning Schild [this message]
2021-03-10 17:30 ` Silvano Cirujano Cuesta
2021-03-10 16:28 ` Henning Schild
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210310182514.25b0c08b@md1za8fc.ad001.siemens.net \
--to=henning.schild@siemens.com \
--cc=amikan@ilbers.de \
--cc=isar-users@googlegroups.com \
--cc=silvano.cirujano-cuesta@siemens.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox