From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6901194598360547328 X-Received: by 2002:a19:6c2:: with SMTP id 185mr755802lfg.143.1606813990692; Tue, 01 Dec 2020 01:13:10 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a19:ccc2:: with SMTP id c185ls922605lfg.3.gmail; Tue, 01 Dec 2020 01:13:09 -0800 (PST) X-Google-Smtp-Source: ABdhPJwzahslfFv/jOku3UrMF35D1NMzFaiF0q9EdqqCsIr5KJjC4+6+YaBLuXgc8iLOHGftQrZ6 X-Received: by 2002:a05:6512:1056:: with SMTP id c22mr758651lfb.179.1606813989678; Tue, 01 Dec 2020 01:13:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1606813989; cv=none; d=google.com; s=arc-20160816; b=st2OLdSWjPQHWbTU9Uiyhur5EQ/uoo1T26YuyRxANGpSUgXsB3YH1OpRmYQNjXLyyD nGP+LncIwKLFQvV+9ffdD4dtxtHpaTFcyGnPedl4FJ4Rq1+9Ns5PklaDbWOkJNRlxaPA oAO8kj7WOyy1EVQRZQRRMGeWIGYtJVX0D4yQyFU9wkF9kISYiJCMWIvnkQ4C6HUPygls GkyypKXjq7XWYkQDubwKedMIRKesmKvLcjrBWZwvQQZzA2HiKwJKSitIy/5qGH7i4tqq uPHEQC5nlYd3F5VRaSDpNovkF14+x3O/sYIvaRRuHA6q4gNdigW5IxmPDz7tDUDuNAsS hyYQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=x8LemWuEY37cx6W9GH043FEpeMDRwmIQtMwMEo0WdW8=; b=oScTjxKABRn7zopvvNFK14ty036slV5Qi7zpziADlltmO3hJrejKGxgA2f5pX3vGix SSV7RkC0NfvKPM5gAg/onD2qDxPFDjlO6svVqwuB+WQjsHUa1FHLv5Dlsseg9PF2fYu0 V2Mg0anAhn9H8PFu8IMXx4olVQ0jDJ3pfmuztXrT3BQdVkitxsqCr5jo1r0vuf/MmGKe TZ+7M/goNm2BWBBwJSpWF6EgyQngbj0T2fuISORYT+pj2K+8yHBLUxmVlw2evSdQLFk6 efNcS5ZA6TRG0Gg7BaRDb2LfWi6i7NHQZXQiKD0+df83F2SE46pkdsPGYw1NkQsyoAgO UtgA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from goliath.siemens.de (goliath.siemens.de. [192.35.17.28]) by gmr-mx.google.com with ESMTPS id i12si47633lfl.0.2020.12.01.01.13.09 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 01 Dec 2020 01:13:09 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) client-ip=192.35.17.28; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id 0B19D8bc020084 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 1 Dec 2020 10:13:08 +0100 Received: from md1za8fc.ad001.siemens.net ([139.22.122.36]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 0B19D89D024048; Tue, 1 Dec 2020 10:13:08 +0100 Date: Tue, 1 Dec 2020 10:13:07 +0100 From: Henning Schild To: Cedric Hombourger Cc: , Subject: Re: [RFC] Image templates Message-ID: <20201201101307.296d4029@md1za8fc.ad001.siemens.net> In-Reply-To: <6bf9a786-51b8-2430-90f5-2857746d003f@mentor.com> References: <6bf9a786-51b8-2430-90f5-2857746d003f@mentor.com> X-Mailer: Claws Mail 3.17.8 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: jy/8puQJ0nne Did not even read it ... strong reject! This smells like a mechanism to keep funny layers non OSS. While allowing others to extend them ... without having their sources. Leading to workaround hacks ... and issues in those "secret" layers not being solved at their core. I would first remove all "mel-*" packages before adding my own ;) ... but i would be stuck with what postinst did break ... because they likely dont prerm ... Maybe i do not get it, you are jumping from a pretty "lame" motivation right into implementation. Let us look at the problems we currently have, who has them, why they have them ... and later see what we could do. I seriously doubt the idea that a "product" can be built by adding a few packages to a base image. A prototype or dev image is more likely ... And imagine yourself bumping a layer onto a new base-binary ... say after a buster-bullseye, NetworkManager->systemd-networkd, cron->systemd-timers jump ... good luck writing/reading that Changelog Henning Am Tue, 1 Dec 2020 08:59:20 +0100 schrieb Cedric Hombourger : > Image Templates for ISAR > ======================== > > Introduction > ------------ > > Embedded System projects may have multiple teams working at different > levels. It is not uncommon to have groups creating a "base platform" > which is handed over to product teams to customize further. Thanks to > ISAR leveraging bitbake, image recipes from a base layer may be > easily extended by product layers but also with its ability to > produce and use a `base-apt` instead of upstream package feeds. This > workflow functions well in many cases, it may not be optimal for > application teams. Such teams are typically receiving a base platform > release that is likely to be left untouched for weeks. Creating a > product image for those teams is sometimes as easy as adding a few > packages. > > In such cases, it may be desirable to have a mechanism for ISAR to > construct product images starting from the base platform root > file-system instead of ISAR's bootstrap root file-system. We will > refer to these base platform root file-systems as _Template Images_. > > This document will describe both the construction and the use of > _template images_. > > Construction > ------------ > > Developers needing to produce image templates in addition to bootable > system images will have their image recipes `inherit` the > `produce-image-template` class either as part of their image > development or by extending an existing image with an appropriate > `.bbappend`. > > The class would invoke a `do_rootfs_template` task after > `do_rootfs_install` and before the post-processing tasks (which > should be applied on the final image). This new task would simply > create a template image in the deploy folder built from the common > root file-system artifacts. Some systems may have additional, more > extensive post-processing operations that must be performed on the > final image, this class will still allow for such operations to > function normally. > > It is anticipated that configuration files in > `/etc/apt/sources.list.d` and `/etc/resolv.conf` should be excluded > (they should be recreated while the template image is imported). A > subsequent update will define a criteria for determining which files > should be excluded from the tarball. As yet the specific mechanism > has not been determined, whether it would be a file list, pattern or > some other external attributes. We anticipate other objects that may > need to be excluded are `/etc/hosts` and proxy configuration, to name > two simple examples. > > The process of constructing the base image would then continue as > usual (with post-processing and imager tasks). > > It is suggested to expose the produced image template as an opaque > binary blob and use a `.tmpl.img` file extension although the initial > implementation is anticipated to be a compressed tarball. The > template filesystem may need embedded meta-data to be consumed by > import mechanism or image recipes that make use of templates but > omitted from the final rootfs. It would be useful at the outset to > either specify a minimal set of metadata (build date, image name and > feature list is the initial list to be considered) and the structure > of that metadata (currently considering either JSON-formatted > suitable for extraction with `jq` or `deb` packaging, which has > embedded metadata in the cpio archive). > > Usage > ----- > > Application teams may create image recipes referencing image > templates in their SRC_URI list. As noted above, image templates > would use the `.tmpl.img` file extension. This would be used by > ISAR's `rootfs` class to locate image templates in the list and make > sure that only one template is specified (extracting a template over > another would have unpredictable results). > > If the bulk of the template intelligence is present in the > `do_rootfs` stages, the `do_rootfs_prepare` task from the `rootfs` > class would behave differently. In this case, an image template may > be added to an image SRC_URI. Instead of copying ISAR's bootstrap > rootfs, the template feature would instead unpack the specified > template. It would also be necessary to copy > `/etc/apt/sources.list.d` from the bootstrap and possibly other files > (e.g. `/etc/resolv.conf`). As the environment may include different > apt sources, a complete `apt-get update` would be in order. > > ISAR would then "enter" the unpacked template carrying updated apt > sources to install packages listed in `IMAGE_INSTALL` and > `IMAGE_PREINSTALL` using `apt- get install` as it does today in > `do_rootfs_install`. > > This design should allow images constructed from an image template to > produce a new image template (e.g. a base platform could provide a > template having a BSP and a minimal Debian system, and middleware > teams could then produce their own template with middleware stacks > added and finally consumed by application teams to produce a final > system image). > > Future Work > ----------- > > It may be desirable to offer a mechanism to remove packages coming > from an image template (application teams may need to optimize the > final system image to meet the footprint constraints of their design > and/or to reduce the attack surface by removing services they do not > need). > > Status > ------ > > This high-level design was created to seek early feedback from the > ISAR developer community by exposing the problem we are trying to > solve and the mechanism we are proposing as a solution. The next step > would be the development of a Proof-of-Concept in the event where > positive comments are received. This document may also be updated as > feedback is received and may result in different approaches being > discussed. > > We will be happy to answer questions you may have or provide more > details in areas where this document would be too evasive. >