* [PATCH v7 1/5] classes: add root filesystem containerizing class
2021-03-12 19:58 [PATCH v7 0/5] Add support for containerized root filesystems Silvano Cirujano Cuesta
@ 2021-03-12 19:58 ` Silvano Cirujano Cuesta
2021-03-12 19:58 ` [PATCH v7 2/5] classes: add new image type 'container-img' Silvano Cirujano Cuesta
` (3 subsequent siblings)
4 siblings, 0 replies; 11+ messages in thread
From: Silvano Cirujano Cuesta @ 2021-03-12 19:58 UTC (permalink / raw)
To: isar-users
This class can be used to create container images which root filesystem
is that generated by the do_rootfs task.
Containerized root filesystems have following possible use-cases:
- Using ISAR as a container image builder.
- Simplify distribution of runtime rootfs (binaries, libraries,
configurations, ...) for application development or testing.
- Distributing SDKs.
Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com>
---
.../classes/image-container-extension.bbclass | 82 +++++++++++++++++++
meta/classes/image.bbclass | 1 +
2 files changed, 83 insertions(+)
create mode 100644 meta/classes/image-container-extension.bbclass
diff --git a/meta/classes/image-container-extension.bbclass b/meta/classes/image-container-extension.bbclass
new file mode 100644
index 0000000..f693627
--- /dev/null
+++ b/meta/classes/image-container-extension.bbclass
@@ -0,0 +1,82 @@
+# This software is a part of ISAR.
+# Copyright (C) Siemens AG, 2021
+#
+# SPDX-License-Identifier: MIT
+#
+# This class extends the image.bbclass for containerizing the root filesystem.
+
+CONTAINER_FORMATS ?= "docker-archive"
+IMAGE_INSTALL += "isar-exclude-docs isar-disable-apt-cache"
+
+containerize_rootfs() {
+ local cmd="/bin/dash"
+ local empty_tag="empty"
+ local full_tag="latest"
+ local oci_img_dir="${WORKDIR}/oci-image"
+ local rootfs="$1"
+ local rootfs_id="$2"
+ local container_formats="$3"
+
+ # prepare OCI container image skeleton
+ bbdebug 1 "prepare OCI container image skeleton"
+ rm -rf "${oci_img_dir}"
+ sudo umoci init --layout "${oci_img_dir}"
+ sudo umoci new --image "${oci_img_dir}:${empty_tag}"
+ sudo umoci config --image "${oci_img_dir}:${empty_tag}" \
+ --config.cmd="${cmd}"
+ sudo umoci unpack --image "${oci_img_dir}:${empty_tag}" \
+ "${oci_img_dir}_unpacked"
+
+ # add root filesystem as the flesh of the skeleton
+ sudo cp -a "${rootfs}"/* "${oci_img_dir}_unpacked/rootfs/"
+ # clean-up temporary files
+ sudo find "${oci_img_dir}_unpacked/rootfs/tmp" -mindepth 1 -delete
+
+ # pack container image
+ bbdebug 1 "pack container image"
+ sudo umoci repack --image "${oci_img_dir}:${full_tag}" \
+ "${oci_img_dir}_unpacked"
+ sudo umoci remove --image "${oci_img_dir}:${empty_tag}"
+ sudo rm -rf "${oci_img_dir}_unpacked"
+
+ # no root needed anymore
+ sudo chown --recursive $(id -u):$(id -g) "${oci_img_dir}"
+
+ # convert the OCI container image to the desired format
+ image_name="isar-${rootfs_id}"
+ for image_type in ${CONTAINER_FORMATS} ; do
+ image_archive="${DEPLOY_DIR_IMAGE}/${rootfs_id}-${image_type}.tar"
+ bbdebug 1 "Creating container image type: ${image_type}"
+ case "${image_type}" in
+ "docker-archive" | "oci-archive")
+ if [ "${image_type}" = "oci-archive" ] ; then
+ target="${image_type}:${image_archive}:latest"
+ else
+ target="${image_type}:${image_archive}:${image_name}:latest"
+ fi
+ rm -f "${image_archive}" "${image_archive}.xz"
+ bbdebug 2 "Converting OCI image to ${image_type}"
+ skopeo --insecure-policy copy \
+ "oci:${oci_img_dir}:${full_tag}" "${target}"
+ bbdebug 2 "Compressing image"
+ xz -T0 "${image_archive}"
+ ;;
+ "oci")
+ tar --create --xz --directory "${oci_img_dir}" \
+ --file "${image_archive}.xz" .
+ ;;
+ "docker-daemon" | "containers-storage")
+ if [ -f /.dockerenv ] || [ -f /run/.containerenv ] ; then
+ die "Adding the container image to a container runtime (${image_type}) not supported if running from a container (e.g. 'kas-container')"
+ fi
+ skopeo --insecure-policy copy \
+ "oci:${oci_img_dir}:${full_tag}" \
+ "${image_type}:${image_name}:latest"
+ ;;
+ *)
+ die "Unsupported format for containerize_rootfs: ${image_type}"
+ ;;
+ esac
+ done
+}
+
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index eddc444..ec93cab 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -76,6 +76,7 @@ inherit image-tools-extension
inherit image-postproc-extension
inherit image-locales-extension
inherit image-account-extension
+inherit image-container-extension
# Extra space for rootfs in MB
ROOTFS_EXTRA ?= "64"
--
2.30.1
^ permalink raw reply [flat|nested] 11+ messages in thread* [PATCH v7 2/5] classes: add new image type 'container-img'
2021-03-12 19:58 [PATCH v7 0/5] Add support for containerized root filesystems Silvano Cirujano Cuesta
2021-03-12 19:58 ` [PATCH v7 1/5] classes: add root filesystem containerizing class Silvano Cirujano Cuesta
@ 2021-03-12 19:58 ` Silvano Cirujano Cuesta
2021-03-12 19:58 ` [PATCH v7 3/5] sdk: add support for containerized sdk Silvano Cirujano Cuesta
` (2 subsequent siblings)
4 siblings, 0 replies; 11+ messages in thread
From: Silvano Cirujano Cuesta @ 2021-03-12 19:58 UTC (permalink / raw)
To: isar-users
Add a new "image" class for generating a container image containing the
target root filesystem.
Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com>
---
meta/classes/container-img.bbclass | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
create mode 100644 meta/classes/container-img.bbclass
diff --git a/meta/classes/container-img.bbclass b/meta/classes/container-img.bbclass
new file mode 100644
index 0000000..79ef3e8
--- /dev/null
+++ b/meta/classes/container-img.bbclass
@@ -0,0 +1,18 @@
+# This software is a part of ISAR.
+# Copyright (C) Siemens AG, 2021
+#
+# SPDX-License-Identifier: MIT
+#
+# This class provides the task 'containerize_rootfs'
+# to create container images containing the target rootfs.
+
+do_container_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
+do_container_image[vardeps] += "CONTAINER_FORMATS"
+do_container_image(){
+ rootfs_id="${DISTRO}-${DISTRO_ARCH}"
+
+ bbdebug 1 "Generate container image in these formats: ${CONTAINER_FORMATS}"
+ containerize_rootfs "${IMAGE_ROOTFS}" "${rootfs_id}" "${CONTAINER_FORMATS}"
+}
+
+addtask container_image before do_image after do_image_tools
--
2.30.1
^ permalink raw reply [flat|nested] 11+ messages in thread* [PATCH v7 3/5] sdk: add support for containerized sdk
2021-03-12 19:58 [PATCH v7 0/5] Add support for containerized root filesystems Silvano Cirujano Cuesta
2021-03-12 19:58 ` [PATCH v7 1/5] classes: add root filesystem containerizing class Silvano Cirujano Cuesta
2021-03-12 19:58 ` [PATCH v7 2/5] classes: add new image type 'container-img' Silvano Cirujano Cuesta
@ 2021-03-12 19:58 ` Silvano Cirujano Cuesta
2021-03-12 19:58 ` [PATCH v7 4/5] docs: document creation of container images Silvano Cirujano Cuesta
2021-03-12 19:58 ` [PATCH v7 5/5] ci: add container image sample configurations Silvano Cirujano Cuesta
4 siblings, 0 replies; 11+ messages in thread
From: Silvano Cirujano Cuesta @ 2021-03-12 19:58 UTC (permalink / raw)
To: isar-users
Extend also task "populate_sdk" to support the creation of a container image
containing the SDK.
Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com>
---
meta/classes/image-sdk-extension.bbclass | 42 ++++++++++++++++++++----
1 file changed, 35 insertions(+), 7 deletions(-)
diff --git a/meta/classes/image-sdk-extension.bbclass b/meta/classes/image-sdk-extension.bbclass
index a8c708a..426b925 100644
--- a/meta/classes/image-sdk-extension.bbclass
+++ b/meta/classes/image-sdk-extension.bbclass
@@ -6,11 +6,25 @@
# This class extends the image.bbclass to supply the creation of a sdk
SDK_INCLUDE_ISAR_APT ?= "0"
+SDK_FORMATS ?= "tar-xz"
+
+sdk_tar_xz() {
+ # Copy mount_chroot.sh for convenience
+ sudo cp ${SCRIPTSDIR}/mount_chroot.sh ${SDKCHROOT_DIR}
+
+ # Create SDK archive
+ cd -P ${SDKCHROOT_DIR}/..
+ sudo tar --transform="s|^rootfs|sdk-${DISTRO}-${DISTRO_ARCH}|" \
+ -c rootfs | xz -T0 > ${DEPLOY_DIR_IMAGE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz
+ bbdebug 1 "SDK rootfs available in ${DEPLOY_DIR_IMAGE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz"
+}
do_populate_sdk[stamp-extra-info] = "${DISTRO}-${MACHINE}"
do_populate_sdk[depends] = "sdkchroot:do_build"
-do_populate_sdk[vardeps] += "SDK_INCLUDE_ISAR_APT"
+do_populate_sdk[vardeps] += "SDK_INCLUDE_ISAR_APT SDK_FORMATS"
do_populate_sdk() {
+ local sdk_container_formats=""
+
if [ "${SDK_INCLUDE_ISAR_APT}" = "1" ]; then
# Copy isar-apt with deployed Isar packages
sudo cp -Trpfx ${REPO_ISAR_DIR}/${DISTRO} ${SDKCHROOT_DIR}/isar-apt
@@ -48,12 +62,26 @@ do_populate_sdk() {
done
done
- # Copy mount_chroot.sh for convenience
- sudo cp ${SCRIPTSDIR}/mount_chroot.sh ${SDKCHROOT_DIR}
+ # separate SDK formats: TAR and container formats
+ for sdk_format in ${SDK_FORMATS} ; do
+ case ${sdk_format} in
+ "tar-xz")
+ sdk_tar_xz
+ ;;
+ "docker-archive" | "oci" | "oci-archive" | "docker-daemon" | "containers-storage")
+ sdk_container_formats="${sdk_container_formats} ${sdk_format}"
+ ;;
+ *)
+ die "unsupported SDK format specified: ${sdk_format}"
+ ;;
+ esac
+ done
- # Create SDK archive
- cd -P ${SDKCHROOT_DIR}/..
- sudo tar --transform="s|^rootfs|sdk-${DISTRO}-${DISTRO_ARCH}|" \
- -c rootfs | xz -T0 > ${DEPLOY_DIR_IMAGE}/sdk-${DISTRO}-${DISTRO_ARCH}.tar.xz
+ # generate the SDK in all the desired container formats
+ if [ -n "${sdk_container_formats}" ] ; then
+ bbnote "Generating SDK container in ${sdk_container_formats} format"
+ containerize_rootfs "${SDKCHROOT_DIR}" "sdk-${DISTRO}-${DISTRO_ARCH}" "${sdk_container_formats}"
+ fi
}
+
addtask populate_sdk after do_rootfs
--
2.30.1
^ permalink raw reply [flat|nested] 11+ messages in thread* [PATCH v7 4/5] docs: document creation of container images
2021-03-12 19:58 [PATCH v7 0/5] Add support for containerized root filesystems Silvano Cirujano Cuesta
` (2 preceding siblings ...)
2021-03-12 19:58 ` [PATCH v7 3/5] sdk: add support for containerized sdk Silvano Cirujano Cuesta
@ 2021-03-12 19:58 ` Silvano Cirujano Cuesta
2021-03-19 16:11 ` Baurzhan Ismagulov
2021-03-12 19:58 ` [PATCH v7 5/5] ci: add container image sample configurations Silvano Cirujano Cuesta
4 siblings, 1 reply; 11+ messages in thread
From: Silvano Cirujano Cuesta @ 2021-03-12 19:58 UTC (permalink / raw)
To: isar-users
Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com>
---
doc/user_manual.md | 127 +++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 127 insertions(+)
diff --git a/doc/user_manual.md b/doc/user_manual.md
index a4f3d1d..f6f49bc 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -19,6 +19,7 @@ Copyright (C) 2016-2019, ilbers GmbH
- [Add a Custom Application](#add-a-custom-application)
- [Enabling Cross-compilation](#isar-cross-compilation)
- [Create an ISAR SDK root filesystem](#create-an-isar-sdk-root-filesystem)
+ - [Create a containerized ISAR SDK root filesystem](#create-a-containerized-isar-sdk-root-filesystem)
- [Creation of local apt repo caching upstream Debian packages](#creation-of-local-apt-repo-caching-upstream-debian-packages)
@@ -84,6 +85,9 @@ If your host is >= buster, also install the following package.
apt install python3-distutils
```
+If you want to generate containerized SDKs, also install the following packages: `umoci` and `skopeo`.
+Umoci is provided by Debian Buster and can be installed with `apt install umoci`, Skopeo is provided by Debian Bullseye/Unstable and has to be installed either manually downloading the DEB and installing it (no other packages required) or with `apt install -t bullseye skopeo` (if unstable/bullseye included in `/etc/apt/sources.list[.d]`).
+
Notes:
* BitBake requires Python 3.4+.
@@ -223,6 +227,54 @@ qemu-system-x86_64 -m 256M -nographic -bios edk2/Build/OvmfX64/RELEASE_*/FV/OVMF
qemu-system-i386 -m 256M -nographic -hda tmp/deploy/images/qemui386/isar-image-base-debian-buster-qemui386.wic.img
```
+### Generate container image with root-filesystem
+
+A runnable container image is generated if you set IMAGE_TYPE to 'container-img'.
+Getting a container image can be the main purpose of an ISAR configuration, but not only.
+A container image created from an ISAR configuration meant for bare-metal or virtual machines can be helpfull to test certain applications which requirements (e.g. libraries) can be easily resolved in a containerized environment.
+
+Container images can be generated in different formats, selected with the variable `CONTAINER_FORMAT`. One or more (whitespace separated) of following options can be given:
+ - `docker-archive`: (default) an archive containing a Docker image that can be imported with [`docker import`](https://docs.docker.com/engine/reference/commandline/import/)
+ - `docker-daemon`: resulting container image is made available on the local Docker Daemon
+ - `containers-storage`: resulting container image is made available to tools using containers/storage back-end (e.g. Podman, CRIO, buildah,...)
+ - `oci-archive`: an archive containing an OCI image, mostly for archiving as seed for any of the above formats
+
+Following formats don't work if running `bitbake ...` (to build the image) from inside of a container (e.g. using `kas-container`): `docker-daemon` and `containers-storage`.
+It's technically possible, but requires making host resources (e.g. the Docker Daemon socket) accessible in the container.
+What can endanger the stability and security of the host.
+
+The resulting container image archives (only for `docker-archive` and `oci-archive`) are made available as `tmp/deploy/images/${MACHINE}/${DISTRO}-${DISTRO_ARCH}-${container_format}.tar.xz` (being `container_format` each one of the formats specified in `CONTAINER_FORMAT`).
+
+### Example
+
+ - Make the relevant environment variables available to the task
+
+For one-shot builds (use `local.conf` otherwise):
+
+```
+export BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE IMAGE_TYPE CONTAINER_FORMAT"
+export IMAGE_TYPE="container-img"
+export CONTAINER_FORMAT="docker-archive"
+```
+
+ - Trigger creation of container image from root filesystem
+
+```
+bitbake mc:qemuarm-buster:isar-image-base
+```
+
+ - Load the container image into the Docker Daemon
+
+```
+xzcat build/tmp/deploy/images/qemuarm/debian-buster-armhf-docker-archive.tar.xz | docker load
+```
+
+ - Run a container using the container image (following commands starting with `#~:` are to be run in the container)
+
+```
+docker run --rm -ti --volume "$(pwd):/build" isar-buster-armhf:latest
+```
+
---
## Terms and Definitions
@@ -834,6 +886,81 @@ ii crossbuild-essential-armhf 12.3 all Inf
~#
```
+## Create a containerized ISAR SDK root filesystem
+
+### Motivation
+
+Distributing and using the SDK root filesystem created following the instructions in "[Create an ISAR SDK root filesystem](#create-an-isar-sdk-root-filesystem)" becomes easier using container images (at least for those using containers anyway)
+A "containerized" SDK adds to those advantages of a normal SDK root filesystem the comfort of container images.
+
+### Approach
+
+Create container image with SDK root filesystem with installed cross-toolchain for target architecture and ability to install already prebuilt target binary artifacts.
+Developer:
+ - runs a container based on the resulting container image mounting the source code to be built,
+ - develops applications for target platform on the container and
+ - leaves the container getting the results on the mounted directory.
+
+### Solution
+
+User specifies the variable `SDK_FORMAT` providing a space-separated list of SDK formats to generate.
+
+Supported formats are:
+ - `tar-xz`: (default) is the non-containerized format that results from following the instructions in "[Create an ISAR SDK root filesystem](#create-an-isar-sdk-root-filesystem)"
+ - `docker-archive`: an archive containing a Docker image that can be imported with [`docker import`](https://docs.docker.com/engine/reference/commandline/import/)
+ - `docker-daemon`: resulting container image is made available on the local Docker Daemon
+ - `containers-storage`: resulting container image is made available to tools using containers/storage back-end (e.g. Podman, CRIO, buildah,...)
+ - `oci-archive`: an archive containing an OCI image, mostly for archiving as seed for any of the above formats
+
+User manually triggers creation of SDK formats for his target platform by launching the task `do_populate_sdk` for target image, f.e.
+`bitbake -c do_populate_sdk mc:${MACHINE}-${DISTRO}:isar-image-base`.
+Packages that should be additionally installed into the SDK can be appended to `SDK_PREINSTALL` (external repositories) and `SDK_INSTALL` (self-built).
+
+Following formats don't work if running `bitbake -c do_populate_sdk ...` (to generate the containerized SDK) from inside of a container (e.g. using `kas-container`): `docker-daemon` and `containers-storage`.
+It's technically possible, but requires making host resources (e.g. the Docker Daemon socket) accessible in the container.
+What can endanger the stability and security of the host.
+
+The resulting SDK formats are archived into `tmp/deploy/images/${MACHINE}/sdk-${DISTRO}-${DISTRO_ARCH}-${sdk_format}.tar.xz` (being `sdk_format` each one of the formats specified in `SDK_FORMATS`).
+The SDK container directory `/isar-apt` contains a copy of isar-apt repo with locally prebuilt target debian packages (for <HOST_DISTRO>).
+One may get into an SDK container and install required target packages with the help of `apt-get install <package_name>:<DISTRO_ARCH>` command.
+The directory with the source code to develop on should be mounted on the container (with `--volume <host-directory>:<container-directory>`) to be able to edit files in the host with an IDE and build in the container.
+
+### Example
+
+ - Make the SDK formats to generate available to the task
+
+For one-shot builds (use `local.conf` otherwise):
+
+```
+export BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE SDK_FORMATS"
+export SDK_FORMATS="docker-archive"
+```
+
+ - Trigger creation of SDK root filesystem
+
+```
+bitbake -c do_populate_sdk mc:qemuarm-buster:isar-image-base
+```
+
+ - Load the SDK container image into the Docker Daemon
+
+```
+xzcat build/tmp/deploy/images/qemuarm/sdk-debian-buster-armhf-docker-archive.tar.xz | docker load
+```
+
+ - Run a container using the SDK container image (following commands starting with `#~:` are to be run in the container)
+
+```
+docker run --rm -ti --volume "$(pwd):/build" isar-sdk-buster-armhf:latest
+```
+
+ - Check that cross toolchains are installed
+
+```
+:~# dpkg -l | grep crossbuild-essential-armhf
+ii crossbuild-essential-armhf 12.3 all Informational list of cross-build-essential packages
+```
+
## Creation of local apt repo caching upstream Debian packages
### Motivation
--
2.30.1
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [PATCH v7 4/5] docs: document creation of container images
2021-03-12 19:58 ` [PATCH v7 4/5] docs: document creation of container images Silvano Cirujano Cuesta
@ 2021-03-19 16:11 ` Baurzhan Ismagulov
2021-03-22 12:48 ` Silvano Cirujano Cuesta
2021-03-26 12:17 ` Silvano Cirujano Cuesta
0 siblings, 2 replies; 11+ messages in thread
From: Baurzhan Ismagulov @ 2021-03-19 16:11 UTC (permalink / raw)
To: isar-users
On Fri, Mar 12, 2021 at 08:58:42PM +0100, Silvano Cirujano Cuesta wrote:
> --- a/doc/user_manual.md
...
> @@ -223,6 +227,54 @@ qemu-system-x86_64 -m 256M -nographic -bios edk2/Build/OvmfX64/RELEASE_*/FV/OVMF
...
> +### Generate container image with root-filesystem
...
> +A container image created from an ISAR configuration meant for bare-metal or virtual machines can be helpfull to test certain applications which requirements (e.g. libraries) can be easily resolved in a containerized environment.
...
> +It's technically possible, but requires making host resources (e.g. the Docker Daemon socket) accessible in the container.
> +What can endanger the stability and security of the host.
Thanks for implementing this long-standing feature, the series looks good to
me.
I'd suggest a couple of cosmetic changes:
* "root-filesystem" -> "root filesystem"
* "ISAR" -> "Isar"
* "container. What can endanger" -> "container, which can endanger"
* Wrap at column 79.
What do you think? Would it be ok to do that when we apply it, or should I send
an updated patch?
We'll play a bit with the images, but I'm quite positive we'll be able to merge
this soon.
With kind regards,
Baurzhan.
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v7 4/5] docs: document creation of container images
2021-03-19 16:11 ` Baurzhan Ismagulov
@ 2021-03-22 12:48 ` Silvano Cirujano Cuesta
2021-03-26 12:17 ` Silvano Cirujano Cuesta
1 sibling, 0 replies; 11+ messages in thread
From: Silvano Cirujano Cuesta @ 2021-03-22 12:48 UTC (permalink / raw)
To: isar-users
On 19/03/2021 17:11, Baurzhan Ismagulov wrote:
> On Fri, Mar 12, 2021 at 08:58:42PM +0100, Silvano Cirujano Cuesta wrote:
>> --- a/doc/user_manual.md
> ...
>> @@ -223,6 +227,54 @@ qemu-system-x86_64 -m 256M -nographic -bios edk2/Build/OvmfX64/RELEASE_*/FV/OVMF
> ...
>> +### Generate container image with root-filesystem
> ...
>> +A container image created from an ISAR configuration meant for bare-metal or virtual machines can be helpfull to test certain applications which requirements (e.g. libraries) can be easily resolved in a containerized environment.
> ...
>> +It's technically possible, but requires making host resources (e.g. the Docker Daemon socket) accessible in the container.
>> +What can endanger the stability and security of the host.
> Thanks for implementing this long-standing feature, the series looks good to
> me.
>
>
> I'd suggest a couple of cosmetic changes:
>
> * "root-filesystem" -> "root filesystem"
> * "ISAR" -> "Isar"
> * "container. What can endanger" -> "container, which can endanger"
> * Wrap at column 79.
>
> What do you think? Would it be ok to do that when we apply it, or should I send
> an updated patch?
I agree with the proposed changes. I'll incorporate them if a new patch version is needed. If not, you can simply update the patch.
>
>
> We'll play a bit with the images, but I'm quite positive we'll be able to merge
> this soon.
I'd be happy to see it happening :-D
BR,
Silvano
>
>
> With kind regards,
> Baurzhan.
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v7 4/5] docs: document creation of container images
2021-03-19 16:11 ` Baurzhan Ismagulov
2021-03-22 12:48 ` Silvano Cirujano Cuesta
@ 2021-03-26 12:17 ` Silvano Cirujano Cuesta
1 sibling, 0 replies; 11+ messages in thread
From: Silvano Cirujano Cuesta @ 2021-03-26 12:17 UTC (permalink / raw)
To: isar-users
On 19/03/2021 17:11, Baurzhan Ismagulov wrote:
> On Fri, Mar 12, 2021 at 08:58:42PM +0100, Silvano Cirujano Cuesta wrote:
>> --- a/doc/user_manual.md
> ...
>> @@ -223,6 +227,54 @@ qemu-system-x86_64 -m 256M -nographic -bios edk2/Build/OvmfX64/RELEASE_*/FV/OVMF
> ...
>> +### Generate container image with root-filesystem
> ...
>> +A container image created from an ISAR configuration meant for bare-metal or virtual machines can be helpfull to test certain applications which requirements (e.g. libraries) can be easily resolved in a containerized environment.
> ...
>> +It's technically possible, but requires making host resources (e.g. the Docker Daemon socket) accessible in the container.
>> +What can endanger the stability and security of the host.
> Thanks for implementing this long-standing feature, the series looks good to
> me.
>
>
> I'd suggest a couple of cosmetic changes:
>
> * "root-filesystem" -> "root filesystem"
> * "ISAR" -> "Isar"
It's not being consistently used in the document. I'll use it as you propose, but the inconsistency will remain.
> * "container. What can endanger" -> "container, which can endanger"
> * Wrap at column 79.
FYI there are enough other areas in the document beyond column 79. I'll apply it nevertheless on my patch.
Silvano
>
> What do you think? Would it be ok to do that when we apply it, or should I send
> an updated patch?
>
>
> We'll play a bit with the images, but I'm quite positive we'll be able to merge
> this soon.
>
>
> With kind regards,
> Baurzhan.
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH v7 5/5] ci: add container image sample configurations
2021-03-12 19:58 [PATCH v7 0/5] Add support for containerized root filesystems Silvano Cirujano Cuesta
` (3 preceding siblings ...)
2021-03-12 19:58 ` [PATCH v7 4/5] docs: document creation of container images Silvano Cirujano Cuesta
@ 2021-03-12 19:58 ` Silvano Cirujano Cuesta
2021-03-22 12:59 ` Anton Mikanovich
4 siblings, 1 reply; 11+ messages in thread
From: Silvano Cirujano Cuesta @ 2021-03-12 19:58 UTC (permalink / raw)
To: isar-users
Add samples for the creation of container images and containerized SDKs
as well as testing configurations to the CI script.
Signed-off-by: Silvano Cirujano Cuesta <silvano.cirujano-cuesta@siemens.com>
---
meta-isar/conf/local.conf.sample | 3 ++
meta-isar/conf/machine/container.conf | 5 ++++
.../conf/multiconfig/container-bullseye.conf | 4 +++
.../conf/multiconfig/container-buster.conf | 4 +++
.../conf/multiconfig/container-stretch.conf | 4 +++
scripts/ci_build.sh | 29 ++++++++++++++++++-
6 files changed, 48 insertions(+), 1 deletion(-)
create mode 100644 meta-isar/conf/machine/container.conf
create mode 100644 meta-isar/conf/multiconfig/container-bullseye.conf
create mode 100644 meta-isar/conf/multiconfig/container-buster.conf
create mode 100644 meta-isar/conf/multiconfig/container-stretch.conf
diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
index 107496c..87ba26f 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -51,6 +51,9 @@ BBMULTICONFIG = " \
qemuamd64-buster \
qemuamd64-buster-tgz \
qemuamd64-bullseye \
+ container-stretch \
+ container-buster \
+ container-bullseye \
qemumipsel-stretch \
qemumipsel-buster \
qemumipsel-bullseye \
diff --git a/meta-isar/conf/machine/container.conf b/meta-isar/conf/machine/container.conf
new file mode 100644
index 0000000..367d790
--- /dev/null
+++ b/meta-isar/conf/machine/container.conf
@@ -0,0 +1,5 @@
+# This software is a part of ISAR.
+# Copyright (C) 2015-2017 ilbers GmbH
+
+IMAGE_TYPE = "container-img"
+
diff --git a/meta-isar/conf/multiconfig/container-bullseye.conf b/meta-isar/conf/multiconfig/container-bullseye.conf
new file mode 100644
index 0000000..200b241
--- /dev/null
+++ b/meta-isar/conf/multiconfig/container-bullseye.conf
@@ -0,0 +1,4 @@
+# This software is a part of ISAR.
+
+MACHINE ?= "container"
+DISTRO ?= "debian-bullseye"
diff --git a/meta-isar/conf/multiconfig/container-buster.conf b/meta-isar/conf/multiconfig/container-buster.conf
new file mode 100644
index 0000000..78b0324
--- /dev/null
+++ b/meta-isar/conf/multiconfig/container-buster.conf
@@ -0,0 +1,4 @@
+# This software is a part of ISAR.
+
+MACHINE ?= "container"
+DISTRO ?= "debian-buster"
diff --git a/meta-isar/conf/multiconfig/container-stretch.conf b/meta-isar/conf/multiconfig/container-stretch.conf
new file mode 100644
index 0000000..3ff8bcb
--- /dev/null
+++ b/meta-isar/conf/multiconfig/container-stretch.conf
@@ -0,0 +1,4 @@
+# This software is a part of ISAR.
+
+MACHINE ?= "container"
+DISTRO ?= "debian-stretch"
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index 3868fb6..a8d861d 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -17,6 +17,15 @@ cd "$(dirname "$0")/.."
# Start build in Isar tree by default
BUILD_DIR=./build
+# Check dependencies
+DEPENDENCIES="umoci skopeo"
+for prog in ${DEPENDENCIES} ; do
+ if [ ! -x "$(which $prog)" ] ; then
+ echo "missing $prog in PATH, exiting" >&2
+ exit 1
+ fi
+done
+
BB_ARGS="-v"
TARGETS_SET="\
@@ -33,7 +42,8 @@ TARGETS_SET="\
mc:qemumipsel-buster:isar-image-base \
mc:nand-ubi-demo-buster:isar-image-ubi \
mc:rpi-stretch:isar-image-base \
- mc:qemuamd64-focal:isar-image-base"
+ mc:qemuamd64-focal:isar-image-base \
+ "
# qemu-user-static of <= buster too old to build that
# mc:qemuarm64-buster:isar-image-base
# mc:qemuarm64-bullseye:isar-image-base
@@ -45,6 +55,12 @@ TARGETS_SET_BULLSEYE="\
mc:qemumipsel-bullseye:isar-image-base \
"
+TARGETS_CONTAINERS="\
+ mc:container-stretch:isar-image-base \
+ mc:container-buster:isar-image-base \
+ mc:container-bullseye:isar-image-base \
+"
+
CROSS_TARGETS_SET="\
mc:qemuarm-stretch:isar-image-base \
mc:qemuarm-buster:isar-image-base \
@@ -237,3 +253,14 @@ bitbake $BB_ARGS mc:qemuamd64-stretch:isar-image-base
mv "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks.ci-backup" "${LAYERDIR_isar}/scripts/lib/wic/canned-wks/sdimage-efi.wks"
mv ${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img.ci-backup \
${BUILDDIR}/tmp/deploy/images/qemuamd64/isar-image-base-debian-stretch-qemuamd64.wic.img
+
+# Finalize with containerized images, since they remove some not-needed packages from the local.conf
+sed -i -e 's/\(IMAGE_INSTALL = .*\) example-module-${KERNEL_NAME}\(.*\)/\1\2/g' conf/local.conf
+sed -i -e 's/\(IMAGE_INSTALL = .*\) enable-fsck\(.*\)/\1\2/g' conf/local.conf
+bitbake $BB_ARGS $TARGETS_CONTAINERS
+while [ -e bitbake.sock ]; do sleep 1; done
+# and SDK container image creation
+SDK_FORMAT="docker-archive" BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE SDK_FORMAT" \
+ bitbake $BB_ARGS -c do_populate_sdk mc:container-stretch:isar-image-base
+while [ -e bitbake.sock ]; do sleep 1; done
+
--
2.30.1
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: [PATCH v7 5/5] ci: add container image sample configurations
2021-03-12 19:58 ` [PATCH v7 5/5] ci: add container image sample configurations Silvano Cirujano Cuesta
@ 2021-03-22 12:59 ` Anton Mikanovich
2021-03-22 13:11 ` Silvano Cirujano Cuesta
0 siblings, 1 reply; 11+ messages in thread
From: Anton Mikanovich @ 2021-03-22 12:59 UTC (permalink / raw)
To: Silvano Cirujano Cuesta, isar-users
12.03.2021 22:58, Silvano Cirujano Cuesta wrote:
> +# and SDK container image creation
> +SDK_FORMAT="docker-archive" BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE SDK_FORMAT" \
> + bitbake $BB_ARGS -c do_populate_sdk mc:container-stretch:isar-image-base
> +while [ -e bitbake.sock ]; do sleep 1; done
> +
Should be SDK_FORMATS here (or SDK_FORMAT in image-sdk-extension.bbclass).
And it's better to add this to local.conf without BB_ENV_EXTRAWHITE.
--
Anton Mikanovich
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: [PATCH v7 5/5] ci: add container image sample configurations
2021-03-22 12:59 ` Anton Mikanovich
@ 2021-03-22 13:11 ` Silvano Cirujano Cuesta
0 siblings, 0 replies; 11+ messages in thread
From: Silvano Cirujano Cuesta @ 2021-03-22 13:11 UTC (permalink / raw)
To: Anton Mikanovich, isar-users
On 22/03/2021 13:59, Anton Mikanovich wrote:
> 12.03.2021 22:58, Silvano Cirujano Cuesta wrote:
>> +# and SDK container image creation
>> +SDK_FORMAT="docker-archive" BB_ENV_EXTRAWHITE="$BB_ENV_EXTRAWHITE SDK_FORMAT" \
>> + bitbake $BB_ARGS -c do_populate_sdk mc:container-stretch:isar-image-base
>> +while [ -e bitbake.sock ]; do sleep 1; done
>> +
>
> Should be SDK_FORMATS here (or SDK_FORMAT in image-sdk-extension.bbclass).
You're right. Good catch!
> And it's better to add this to local.conf without BB_ENV_EXTRAWHITE.
In general yes, but being it the CI script I was just trying to make it somehow more explicit than changing the local.conf.
But since I don't have a strong opinion on it, I don't mind changing it as you propose.
Silvano
^ permalink raw reply [flat|nested] 11+ messages in thread