From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6543174714398867456 X-Received: by 10.46.44.14 with SMTP id s14mr377432ljs.43.1523866624002; Mon, 16 Apr 2018 01:17:04 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 10.46.146.13 with SMTP id k13ls117009ljg.4.gmail; Mon, 16 Apr 2018 01:17:03 -0700 (PDT) X-Google-Smtp-Source: AIpwx48GJ68++lt2hapIiufcbMMmDmjy70OhjGJA075dMOvRAXlxpzkTtDk6dRtaKxdNwCisj43J X-Received: by 10.46.158.81 with SMTP id g17mr73163ljk.34.1523866623427; Mon, 16 Apr 2018 01:17:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1523866623; cv=none; d=google.com; s=arc-20160816; b=zIuW4hnnCKKjw7uCwaCiDaUk5Y6gDvJKdUmiVBb0d/h2RnbstXg8N5sEwWqzsx/PK6 9gbdzqgwtZtd7Q5vFFWIlRsZZcqPq6LRrS+pxTXnh6ee+2QABcMqwBPbS8OsU59Z8wN9 dGxQfiweAIjuGoeJH3mwptEWGKGDRkfJ5HGquF2xgEKEihVWD9eW15KY9IdSmBX+wlVe 2HS0tKuh2sQF8dJuoRYiPSz2x7X7Ar9PEDrcClJ1ZeZ3cuSZYVU83z/3Z6ejdU4I0bWV H5rZVDrQrmQHk+8ffrmzCGErh6Vg48mceWnFg8JD7NRJMdVnRcR4EO2pSYOX9th6u4Vs m5fg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject :arc-authentication-results; bh=g8HzmrXP2KuHa/B+jlMqecSy0mVlmrPQABZVg8DGhds=; b=NLzKjxpLsvY6poMesdRCDXOFLk9Mw12AkTLWS/lCG70w9TZoXSvIm7CMAf51A9gq8R weaMF0neEURUEQ0tvxQFZRQ1aWxGlI63RyP7AuI12ZguqVN7eidkHi2B9fqjlr2qOnRu p5UWmon+J/+VT2ytyhdTeJwvt92ne9NdNu7qvGBTglR95aEg6O53IHAPaaEtoNcTKm5u 6KumWeRLsCGE3DZRhZcd07kgDOa9BvKUECJYMkXICdHKxdBq6MQJ8CEme/1IlO0l5HmQ YjydrMQVtOneErdmwaHuUB4FQO7NtYySigVoGBk6mJ/rQLRF/WKYeBP8eRlUYZjAB6Te Egmw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id t83-v6si375868lff.5.2018.04.16.01.17.03 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 16 Apr 2018 01:17:03 -0700 (PDT) Received-SPF: pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id w3G8H21X017900 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Mon, 16 Apr 2018 10:17:02 +0200 Received: from [139.25.69.226] (linux-ses-ext02.ppmd.siemens.net [139.25.69.226]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id w3G8H2vf011906 for ; Mon, 16 Apr 2018 10:17:02 +0200 Subject: Re: 'IMAGE_FEATURES' in Isar To: isar-users References: <8cc3abfe-de1b-fb63-b58d-7a73561c11ae@siemens.com> <20180412142614.GE6864@yssyq.radix50.net> <9e911992-c9ff-647d-ff04-2a400677d6e9@siemens.com> <20180413112731.GF6488@yssyq.radix50.net> From: Claudius Heine Message-ID: <02703006-308b-a307-9c33-cd0791ed3fdb@siemens.com> Date: Mon, 16 Apr 2018 10:17:02 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <20180413112731.GF6488@yssyq.radix50.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: Cmb7z0cKrGOr Hi Baurzhan, On 2018-04-13 13:27, Baurzhan Ismagulov wrote: > On Thu, Apr 12, 2018 at 05:35:17PM +0200, Claudius Heine wrote: >> Alex brought up the IMAGE_FEATURES idea and this thread just discuss it and >> to answer the questions: What else could IMAGE_FEATURES be used for? And >> does it make sense to use it to specify if the kernel should be part of the >> image or not? > > Ah, now I see your point. > > As I understand it, Alex's point is to introduce an additional, breve, binary > porcelain for the user. Adding or removing hairy IMAGE_INSTALL lines should be > hidden in the plumbing. > > E.g., poky/meta/recipes-sato/images/core-image-sato.bb: > > IMAGE_FEATURES += "splash package-management x11-base x11-sato \ > ssh-server-dropbear hwcodecs" > > When we have that many optional things, copy-pasting them in multiple recipes > would be an error-prone process and a maintenance problem due to duplication of > IMAGE_INSTALL += lines that have all to be updated if anything changes. > > In our case, local.conf would have IMAGE_FEATURES = "kernel" by default. Of > course, we can only go in that direction if something like > IMAGE_FEATURES_isar-image-base = "" would work (maybe it does [1]). It isn't > very Isar-like, since Yocto is focused on one config at a time, and we > explicitly aim at multiple, diverse deliverables from the beginning. Ideally, > I'd like to see such binary porcelain in a recipe (not touching machine and > distro configs being the goal). The problem with porting 'IMAGE_FEATURES' directly to Isar can be read in the OE documentation: IMAGE_FEATURES: Specifies features to include in the image. Most of these features map to additional packages for installation. [1] Of course IMAGE_FEATURES in OE do a bit more that just 'IMAGE_INSTALL += "packagegroup-feature"' or just 'IMAGE_INSTALL += apt'. But that would be the case if we start mapping those features directly to Isar, since using a binary distribution offers less customizability than using a source distribution. The basic idea of IMAGE_FEATURES is to be orthogonal to IMAGE_INSTALL, otherwise having it doesn't make sense. You have to use it for stuff you cannot express via just some entries in IMAGE_INSTALL. The only thing that comes to my mind and is not covered in some other aspect of Isar already, are those currently monolithic configuration scripts. Breaking those scripts up into smaller parts like 'no-package-management', 'no-headers', 'no-manpages', 'no-other-locales', 'no-package-cache' and maybe some other things I currently cannot think of, and executing them after all packages are install, controlled by IMAGE_FEATURES, would be orthogonal to IMAGE_INSTALL entries and would provide value. Yes, those 'no-*' features aren't OE like, but this is a concession to the binary distribution Isar is based on. Also to the idea of OVERRRIDES, we currently don't use them at all, and we should add at least the MACHINE, DISTRO, DISTRO_ARCH and MULTICONFIG to it, just to be able to better select different configurations in bbappend files. About your idea with adding the image recipe name to it, I don't know if that is even possible. >>>> - Cleanup steps like localepurge, and apt-get clean >>> >>> Just do it always, as in your patches? >> >> You meant the '*-configscripts.sh'? >> >> AFAIK the purpose is to try to move most of this stuff out of there and into >> a 'system-setup-package' of sorts. At lease for all of those things, that >> don't require doing it at the end of everything. > > Agree with the latter (e.g., root password, hostname). > > Do I understand correctly that localepurge and apt-get clean have to be done at > the end? If yes, then they aren't suitable for system-setup-package. Then, > where should those go? As I described above: Into IMAGE_FEATURES, since those steps are orthogonal to IMAGE_INSTALL or packages in general. > Regarding whether that should be done in IMAGE_FEATURES, my opinion is still > that specifically localepurge, apt-get clean should always be performed. IOW, I > don't think they belong to IMAGE_FEATURES. I am not so sure about that. For instance the 'buildchroot' is also basically just an image, where those steps aren't necessary. But maybe deploying the 'build.sh' script is a IMAGE_FEATURE for buildchroot. We cannot install it as a package, since the required buildchroot for packaging it is not done yet. >> The rest of those steps (like those cleanup commands) should be split up >> into smaller scripts each with just one task and then trigger them via >> IMAGE_FEATURES after all packages were installed. Then the difference >> between raspbian and debian might just be some different entries in the >> IMAGE_FEATURES (or DISTRO_FEATURES) variable. At least thats my current idea >> here. > > We can discuss those when we have a specific example. I just took a look at what the current configscripts are doing. Here are my basic thoughts: - install /etc/default/locales > package - configuration of localepurge > package - debconf settings > Don't know yet, maybe package? - install /etc/fstab > package or wic - create devices > package - /etc/inittab > package - execute localepurge > feature that also depends on package - apt-get clean > feature - setting boot cfg > package or wic - remove qemu-*-static > feature (since buildchroot doesn't need it) > > >>>> - Removal of apt and other possible reduction tasks >>> >>> Additional recipe some images depend on? >> >> Removal of apt and other reduction steps should be done after all packages >> are installed using apt. Doing that in some other recipe/debian package just >> doesn't work. > > Ok, so it has to be a task. Yocto does it by default and provides > IMAGE_FEATURES += "package-management". I think mimicking that would be an > advantage here. As I said further up: OE can do stuff like 'package-management' feature because they start with a completly clean state and specifying only stuff to add is possible there. With Isar we start with a minimal base system, that already includes a package-managment per default. A feature to have it doesn't make sense, a feature to remove it does IMO. >>>> The other question might be if having a kernel should be implemented as a >>>> IMAGE_FEATURE or rather a setting in the machine.conf? > > I'd like to provide a binary way to customize it for every image without > touching machine or distro confs. Ok. I get it. From were I came from, I used 'having a kernel' and 'not having a kernel' as a function of the machine configuration or multiconfig configuration. But you see it as a function of the image. Since adding the kernel to IMAGE_INSTALL or evaluating IMAGE_FEATURES is mostly done in the specific image, it can also be changed there. >>> We already have it in IMAGE_INSTALL for isar-image-base. Doesn't it work if the >>> kernel is removed from there? >> >> This is how I do it in the current "Optional kernel" patchset. > > After I see the implementation, I think stacking complex lines in multiple > images has disadvantages compared to binary options. I don't think I understand you here. Do you mean that you prefer having: IMAGE_INSTALL += "${@ ("linux-image-" + d.getVar("KERNEL_NAME", True)) if 'kernel' in d.getVar("IMAGE_FEATURES", True) else ""}" instead of: IMAGE_INSTALL += "${@ ("linux-image-" + d.getVar("KERNEL_NAME", True)) if d.getVar("KERNEL_NAME", True) else ""}" If that is your issue, than ok, we can change that. But as I said, I don't think that IMAGE_FEATURES should work that way, since that way they are just a simple mapping of string -> string. Meaning a string in IMAGE_FEATURES results in a string in IMAGE_INSTALL. And we miss out of using IMAGE_FEATURES in a way orthogonal to IMAGE_INSTALL. Cheers, Claudius [1] https://www.yoctoproject.org/docs/latest/mega-manual/mega-manual.html#image-generation-dev-environment -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de