From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6901194598360547328 X-Received: by 2002:a05:651c:1122:: with SMTP id e2mr1139243ljo.317.1606998242501; Thu, 03 Dec 2020 04:24:02 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a19:ccc2:: with SMTP id c185ls309465lfg.3.gmail; Thu, 03 Dec 2020 04:24:01 -0800 (PST) X-Google-Smtp-Source: ABdhPJw/8Np7JxYjfXBr3pkC2lUNYrxQFZptgNkpp5vpDdJkpfyc96nHu+qNuifrJW5h08WG26Ad X-Received: by 2002:a19:151:: with SMTP id 78mr1243436lfb.170.1606998241390; Thu, 03 Dec 2020 04:24:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1606998241; cv=none; d=google.com; s=arc-20160816; b=CILiNUMyIP5MpjqR0irKzyUotenlkaKWtoIGPR142QUv7jbERJ0eqYIeh4jX5MEuHi q8KEy3M3Tk8YHFCowQbGEKlK4wW9IN/tLrMJT9e5q7d7ZREs5DIDo0gTrJ2xu6sYL8at eVOZvSwVALhdPSB2eHB0y9GOu81x0WY2dlWnYQSmj2reA+sxY5TouhqYO6WR7gVJjwGr Npi77n9WJYbHMiGAbRbXiw32azRlEq1E7Zp5kYum1AEmDfxkvn6OkwyBaLzZXjkjYbZD vUeKlUel9tNmftBCCiWEDY+KRNDhqQaRBPxw8+CuamldTsoWXcB/ucTGn+MF6LRy/BCr zuwQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:organization:from:references:cc:to :subject; bh=BlSS3dSh6m5MV4DfGxis1yA6Zz1lFOHPOyOodrRySaM=; b=gIL4+8AsuZW85toVDWNIysrzsJd10Ei0gFZwvZfzi7ojmK5Ch1tdsY3JCfeNLXfUnL 4OcdJicGUTZjMukIQYKrPk3zXsL0GXsqZuSa9/FCQjxopWAPDMWLFmqEc3qlgi7X1lNX UGOLiBtMoJnbWq9T+cmySPG/ocBmNcSHcVfaCnR583pbHD56360GNpRTx62FPIjwewNz jWTGmQhnwg/88RBl/sndMLSJGBOD0Xwaj+SVF0RZrLmoME44p/nwXRv2Hv9Oj+Fz7vjz k8kHaosRrlJ2dYRVAU/faUCWm7eL32LKgzDHUC9SV57liaRzGLM2/gNWg5XKLIyYfua2 C8dQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=neutral (google.com: 212.18.0.10 is neither permitted nor denied by best guess record for domain of ch@denx.de) smtp.mailfrom=ch@denx.de Return-Path: Received: from mail-out.m-online.net (mail-out.m-online.net. [212.18.0.10]) by gmr-mx.google.com with ESMTPS id m18si47298lfr.11.2020.12.03.04.24.01 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Thu, 03 Dec 2020 04:24:01 -0800 (PST) Received-SPF: neutral (google.com: 212.18.0.10 is neither permitted nor denied by best guess record for domain of ch@denx.de) client-ip=212.18.0.10; Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 212.18.0.10 is neither permitted nor denied by best guess record for domain of ch@denx.de) smtp.mailfrom=ch@denx.de Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 4Cmw4J56Qtz1s2CN; Thu, 3 Dec 2020 13:24:00 +0100 (CET) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 4Cmw4J4nXYz1qy6H; Thu, 3 Dec 2020 13:24:00 +0100 (CET) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id Q_l_DkhZc7gj; Thu, 3 Dec 2020 13:23:59 +0100 (CET) X-Auth-Info: c23Z86OTBXmQ98N/auqDB4qvzj9Xum02jrd7hbqLRd0= Received: from [10.88.0.186] (dslb-084-056-254-149.084.056.pools.vodafone-ip.de [84.56.254.149]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPSA; Thu, 3 Dec 2020 13:23:59 +0100 (CET) Subject: Re: [RFC] Image templates To: Joe MacDonald Cc: Cedric Hombourger , isar-users@googlegroups.com References: <6bf9a786-51b8-2430-90f5-2857746d003f@mentor.com> <20201202163731.GC7804@mentor.com> From: Claudius Heine Organization: Denx Software Engineering Message-ID: <3ad50365-6298-8ba3-f2ce-39f5f6210fe9@denx.de> Date: Thu, 3 Dec 2020 13:23:58 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.5.0 MIME-Version: 1.0 In-Reply-To: <20201202163731.GC7804@mentor.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: eDiUKbkzLTfe Hi Joe, On 2020-12-02 17:37, Joe MacDonald wrote: > Hi Claudius, > > [Re: [RFC] Image templates] On 20.12.02 (Wed 14:00) Claudius Heine wrote: > >> Hi Cedric, >> >> On 2020-12-01 08:59, Cedric Hombourger wrote: >>> Image Templates for ISAR >>> ======================== >>> >>> Introduction >>> ------------ >>> >>> Embedded System projects may have multiple teams working at different >>> levels. It is not uncommon to have groups creating a "base platform" >>> which is handed over to product teams to customize further. >>> Thanks to ISAR leveraging bitbake, image recipes from a base layer may >>> be easily extended by product layers but also with its ability to >>> produce and use a `base-apt` instead of upstream package feeds. >>> This workflow functions well in many cases, it may not be optimal for >>> application teams. Such teams are typically receiving a base platform >>> release that is likely to be left untouched for weeks. >>> Creating a product image for those teams is sometimes as easy as adding >>> a few packages. >>> >>> In such cases, it may be desirable to have a mechanism for ISAR to >>> construct product images starting from the base platform root >>> file-system instead of ISAR's bootstrap root file-system. We will refer >>> to these base platform root file-systems as _Template Images_. >> >> In principle I understand your requirement, however I don't think Isar (and >> possible Debian) is the right base for this. >> >> What you probably require is some sort of functional and non-destructive >> root-fs build approach to root file system, that lets you arbitrarily cache >> and distribute each stage of the root fs build. Generating and reusing the >> 'base-apt' repository is much easier than it would be for root file systems. >> >> Also stuff like docker solves only a small part of that solution. >> >> You will probably have to build a new system inspired by stuff like the >> bazel build system[1], and nixox[2] or guixos[3] and maybe even stuff like >> ostree [4]. > > Thank you for the suggestions. I had looked at bazel a while back for > something unrelated but my impression of it is really that it's aimed > squarely at application development teams, so never really investigated > how they do their builds. The list is more for inspiration sake, not to actually use any of those technologies. You are right Bazel is not really useful for this kind of work. I also don't have much experience with this, but AFAIK they are more of a Copy-on-Write build system. Which might be a good idea, because that would allow you to share partial build products. > We had a proof-of-concept working about a year > ago with ostree on top of Isar but ultimately that didn't seem to be a > good fit either since it placed significant restrictions on the > filesystem. In the intervening time the project has also moved in a more > container-focused direction, so it also doesn't seem to address this space > any longer. Again my point was not using ostree directly, but to let you be inspired by it. What they did was putting a version of root file systems, that lets you reuse parts and create small deltas. > NixOS and Guix both warrant further investigation, I think, though I've > done a bit of preliminary work with NixOS and I think the differences > between it and Debian in package format and maintenance tools are too > great to make it practical. NixOS lets you download pre-build artifacts at the same time as allowing you to add custom patches to packages, causing it to just rebuild only what you changed, which would again be a pretty great attribut to have for such a system. > In short, I definitely agree this is potentially a significant adjustment > and a large undertaking, but Isar does also seem almost uniquely > positioned to make this work, and why whe're making this proposal and > looking for feedback. Isar is based on Debian, and while Debian is a great general Distribution, I don't think that pick and choosing different rootfs templates with different versions together would be possible. Which would IMO be a central property for such a system. I understand that this proposal only considers a multiple images based on one template (1:m), because doing it otherwise could not really be solved here. But the modularity of bitbake based projects with OE and Isar is about m:n, e.g. multiple layers can be used and reused for multiple images. >> The rest of what you describe sounds like they would try to shoehorn this >> into the existing isar build process and then trying to explicitly fix just >> some of the many future issues you will run into, instead of actually >> designing something that is actually able to deal with all those issues in a >> generic way. > > Are you thinking of something specific in the design that you see being > long-term problematic, or is this more of a general concern like "this is > plainly out of scope for Isar"? Our objective is to produce something > that compliments and extends existing workflows, not to bolt on something > that doesn't make sense, so we're open to all feedback and views of what > could be danger areas or sources for future pain. You will have to deal also with removal of packages that where installed in a template, and possible allow to patch and rebuild packages that are part of the template. That would mean the packages written in other all other isar layers now could cause a whole new dimension of issues. Isar packages don't really have the same QA as packages that are part of Debian and might not work well with removal. (This is also pointed out by Henning) No idea how you would deal with detecting if some packages needs to be rebuild, because a patch was added. You will probably have to deal with signatures and sstate.. > >> Personally, I would be interested in such a new system, but that will take >> serious dedication to design and implement it properly. >> >> Changing that now in Isar will probably break a lot of assumptions and >> therefore limit its usefulness. > > That's definitley not our goal and the approach we are proposing, done as > a new class, intends to ensure it doesn't spider out and touch things that > will impact unrelated Isar workflows. If you can point to areas where > we're potentially breaking assumptions in Isar, most likely around image > assembly, but maybe other oareas, we'd be very happy to refine our view > and goals. - Packages in Isar don't have the same QA as packages in Debian, and might not work with removal, because the assumption was that if they should not be part of an image, they would not have been installed. - Packages that are part of the template might need to be recompiled, because the image itself installed a different version of a library. This is solved in isar, because it would just build everything and let apt figure out the correct dependencies and installs all packages, but with this, you don't really have a way to call back into isar to compile some more packages. (e.g. kernel modules that are part of the template image, and I make a small, stable kernel update in the depending image) I would expect a lot more of those... regards, Claudius