From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6917993065941565440 X-Received: by 2002:a2e:5042:: with SMTP id v2mr682836ljd.297.1611311609573; Fri, 22 Jan 2021 02:33:29 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6512:3993:: with SMTP id j19ls1987158lfu.3.gmail; Fri, 22 Jan 2021 02:33:28 -0800 (PST) X-Google-Smtp-Source: ABdhPJzlGmFIGRpbQrF0ZbcuQg5wHrKOedm0LMHNE0EJCEGfobYteqJVYlDXcR7sjjI8eQyJeIVe X-Received: by 2002:ac2:59d9:: with SMTP id x25mr1860178lfn.39.1611311608442; Fri, 22 Jan 2021 02:33:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1611311608; cv=none; d=google.com; s=arc-20160816; b=PpXcm4Lrs9zWjnZMSl34KvYGee3x1HacZQupwYtPCmQ243b87614k3GMaANg2HuRkq FuIL6RMxcKgNTe+flpZfiLQkx3o6Et7mSFCu7d7TemtM6YXEJVSj2mKub5Uwr/hTvLMl JtzavCThErUFJnF2r+iQ9DAaD12b4xODIG3RA7n7qjNr7UWOjYWo2YgIkTrPgEoC5g4l ulAP+/b443oA1EDedrKTVDE3HptmvdI83qhZI1bSzYb/Mga25zygKQ5eeIX0vWoxg24e F/FAqNm69qakvujh43fM0qD0fuQCqYLwRg1W9jr6fwTADYD5NlcwMUo3cT/RK+EAfvmY Y15Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:organization:from:references:cc:to :subject; bh=weFe+j+7W9nd2VSVrwPeiBeOK0BLc0VnHmB2BnSwDzg=; b=bpDFkoxec2nip8BO2vWSLB1LqduHvuVHPtoEsJVXGu7x2jA3f1jCFXMhSqlTiPQwt+ +pyVf9tE3GD4+9U7VIKCK5aFFpPXivRXlYycKLg/i6ab4jWqqZU7sQAhtClcXrg4UVsI Ef0GiFpNIhLTOfS+UfNuQftIEueOvCyz9+llmCp05AgZ/OROJLNnpHWXiJpqY1nqunIO iFzj5qYwNxNHuhsl7Ezqn01cetDaAEkKcNmEKPvUrFVRVSrlig70IDsSWpfKSzmiHfgx mFc3Y+o9eZy/cCdjWU1YuoBZUZLIyq4n1w+gmAYf2U4Y2MRJDLtwtB4yHa11Ww6IhCd/ 1ttQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=neutral (google.com: 2001:a60:0:28:0:1:25:1 is neither permitted nor denied by best guess record for domain of ch@denx.de) smtp.mailfrom=ch@denx.de Return-Path: Received: from mail-out.m-online.net (mail-out.m-online.net. [2001:a60:0:28:0:1:25:1]) by gmr-mx.google.com with ESMTPS id i18si336654lfp.2.2021.01.22.02.33.28 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 22 Jan 2021 02:33:28 -0800 (PST) Received-SPF: neutral (google.com: 2001:a60:0:28:0:1:25:1 is neither permitted nor denied by best guess record for domain of ch@denx.de) client-ip=2001:a60:0:28:0:1:25:1; Authentication-Results: gmr-mx.google.com; spf=neutral (google.com: 2001:a60:0:28:0:1:25:1 is neither permitted nor denied by best guess record for domain of ch@denx.de) smtp.mailfrom=ch@denx.de Received: from frontend01.mail.m-online.net (unknown [192.168.8.182]) by mail-out.m-online.net (Postfix) with ESMTP id 4DMbFg5G95z1rynf; Fri, 22 Jan 2021 11:33:27 +0100 (CET) Received: from localhost (dynscan1.mnet-online.de [192.168.6.70]) by mail.m-online.net (Postfix) with ESMTP id 4DMbFg4tS7z1tYVg; Fri, 22 Jan 2021 11:33:27 +0100 (CET) X-Virus-Scanned: amavisd-new at mnet-online.de Received: from mail.mnet-online.de ([192.168.8.182]) by localhost (dynscan1.mail.m-online.net [192.168.6.70]) (amavisd-new, port 10024) with ESMTP id kh7BYK4R4KUT; Fri, 22 Jan 2021 11:33:25 +0100 (CET) X-Auth-Info: NFHlijfmc1RqMoV2vmeXmlwI39tVDP5wNSwmQHsByLc= Received: from [10.88.0.186] (dslb-002-207-026-175.002.207.pools.vodafone-ip.de [2.207.26.175]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.mnet-online.de (Postfix) with ESMTPSA; Fri, 22 Jan 2021 11:33:25 +0100 (CET) Subject: Re: image-postproc-extension.bbclass modifying /etc/os-release To: Silvano Cirujano Cuesta , Henning Schild Cc: isar-users@googlegroups.com References: <67e1fac9-5af5-29aa-de57-9a0de0cdd165@siemens.com> <79cdea42-8338-2e7f-33dd-f396db634a14@siemens.com> <20210119092531.2cc80db5@md1za8fc.ad001.siemens.net> <20210119093324.52410271@md1za8fc.ad001.siemens.net> <34c9ad2c-a330-0074-cfd1-bffa1afcbd02@siemens.com> <20210119102246.53791a9c@md1za8fc.ad001.siemens.net> <12759268-1ad7-168a-dde9-4f2658567af1@siemens.com> <402794f7-74a3-ab50-972e-7682d5388ff2@denx.de> <0beb8d2d-5141-647c-a831-8693276c957a@siemens.com> From: Claudius Heine Organization: Denx Software Engineering Message-ID: <333bd498-2e79-bb2d-ef84-3f6ab68a68b9@denx.de> Date: Fri, 22 Jan 2021 11:33:25 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Thunderbird/78.6.1 MIME-Version: 1.0 In-Reply-To: <0beb8d2d-5141-647c-a831-8693276c957a@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-TUID: 3bf1PmnBH1ef Hi Silvano On 2021-01-22 10:47, Silvano Cirujano Cuesta wrote: > Hi Claudius, > > TL;DR: you're proposing here what appears to be a feasible solution to this issue without using a package > > Nevertheless, more information inline. > > On 22/01/2021 09:52, Claudius Heine wrote: >> Hi Silvano, >> >> On 2021-01-19 11:37, Silvano Cirujano Cuesta wrote: >>> We can create a removal file diversion for /usr/lib/os-release (leaving /etc/os-release untouched, that way the Debian manpage for os-release still applies). This approach has following attributes: >>> >>> - /usr/lib/os-release is not managed by any package and that way post-processing it is not an issue and dpkg won't break anything >>> >>> - Debian os-release manpage still applies >>> >>> If going for the other approach (create our own "base-files" package) we can do pretty much the same: >>> >>> - the package doesn't provide /usr/lib/os-release, leaving it package-unmanaged >>> >>> - image post-processing to create the file becomes a requirement >> I looked at the manpage of os-release and it does not state that `/etc/os-release` can or should not be a file: > > That's probably a misunderstanding. I don't mean that we have to create "a file", but provide the os-release information according the specification. There are multiple different valid combinations: > > - /etc/os-release symlink to /usr/lib/os-release > > - /usr/lib/os-relase only (not /etc/os-release at all) > > - /etc/os-release only, no /usr/lib/os-release at all (it's not the recommended way, but it isn't forbidden either) > > - /etc/os-release is a symlink to another file like /usr/lib/isar-release, no /usr/lib/os-release at all (it's not the recommended way, but it isn't forbidden either) Well I had this in mind: - /etc/os-release is a file containing the isar version, /usr/lib/os-release is a file containing the original debian version as deployed by base-files. This could also be done similar to your fourth solution: - /etc/os-release is a symlink to another file like /usr/lib/isar-release, original /usr/lib/os-release from base-files The manual page states that /etc/os-release takes precedence over /usr/lib/os-release, so if /etc/os-release exists, it does not matter if /usr/lib/os-release does as well or what its content is. > > ... > >> >> ``` >>        The file /etc/os-release takes precedence over /usr/lib/os-release. Applications should check for the former, and exclusively use its data if it exists, and only fall back to /usr/lib/os-release if >>        it is missing. Applications should not read data from both files at the same time.  /usr/lib/os-release is the recommended place to store OS release information as part of vendor trees. >>        /etc/os-release should be a relative symlink to /usr/lib/os-release, to provide compatibility with applications only looking at /etc/. A relative symlink instead of an absolute symlink is necessary >>        to avoid breaking the link in a chroot or initrd environment such as dracut. >> >>        os-release contains data that is defined by the operating system vendor and should generally not be changed by the administrator. >> >>        As this file only encodes names and identifiers it should not be localized. >> >>        The /etc/os-release and /usr/lib/os-release files might be symlinks to other files, but it is important that the file is available from earliest boot on, and hence must be located on the root file >>        system. >> ``` >> >> So why not make sure `/etc/os-release` is a file and do this: >> >>     dpkg-divert --add --local --no-rename /etc/os-release >> >> in the post processing step? Shouldn't this prevent any future base-files update from preventing to overwrite the `/etc/os-release`? > > This command would so to say command dpkg not to touch /etc/os-release from the point it's executed on. That way you can have a local version of /etc/os-release and be sure that no package updates is modifying/replacing it. Such a command would go into the postinst hook and another postrm hook would be required to remove the file diversion for /etc/os-release and restoring the original one. Why would such a command go into a postinst hook? It contains '--local' therefore I would expect the administrator doing it him or herself. Or in Isars case a bitbake post installation process. > > This would technically resolve the issue, but looks a bit weird to me. Debian and derivatives get that file managed by a package. But since I haven't understood what the issue mentioned by Henning regarding multiconfig and packages, this might be the only feasible solution... Simple example: Our /etc/os-release contains the build date and git commit id. If we put this information into a package, then it we could not cache it, because this information will change constantly. That also means all dependent tasks and recipes (image creation) cannot be cached, breaking the caching of a lot of stuff. Calling 'bitbake isar-image' two times in short succession without any change, would cause 2 full builds of the whole image. >> The only issue I would see if someone performs a distribution upgrade and now the `/etc/os-release` no longer contains the correct version, but other than an apt.conf hook that updates `/etc/os-release` based on new information from `/usr/lib/os-release` after an upgrade, I don't see a way to do this automatically. > > Why should a distribution upgrade change that? I'm not aware that dist-upgrade changes file diversions... To my knowledge, once the file diversion has been created, dpkg should completely ignore that file. > > File diversions don't apply to packages, but to files, therefore I'd expect it to survive a dist-upgrade. Therefore dpkg doesn't keep track of the packages that are "blocked" from touching a file. It keeps track of the file path and of the package that requested the file diversion (unless "--local" specified, as you propose). You misunderstood me. In my example a base-files package would still update /usr/lib/os-release to contain a new debian suite name, if you do a distribution update. If you want our diverted /etc/os-release to contain this new info automatically, you could do this via a apt.conf.d hook. > >> >> I would rather divert '/etc/os-release' in a postprocessing script than `/usr/lib/os-release` because, I see the post-processing more of a local configuration. > > Diverting /etc/os-release or /usr/lib/os-release depends on what we want to achieve (it would be even better letting the ISAR user decide it): I don't want to divert /usr/lib/os-release. Let it contain the upstream version for reference. This file does not matter, because /etc/os-release takes precedence over it. > > 1. Keep originating Debian for reference? Then change only /etc/os-release. Yes. > > 2. Wipe away all information about originating Debian? Then change /usr/lib/os-release. No. > > /etc/os-release might appear like a local configuration for the ISAR builder, but IMO it's intrinsic information about the distribution being built with ISAR. As the documentation that you are quoting says "defined by the operating system vendor and should generally not be changed by the administrator." Therefore IMO the path /etc/os-release was kept for historical reasons (see footnote 1 of the announcement [1]), but it's an exception to the convention that files under "/etc" are configurations and can be changed. Well its a separate discussion if Isar is in the role of a distribution vendor or of an administrator. I would say its something in between. > > [1] http://0pointer.de/blog/projects/os-release > >> For multiple reasons we cannot used a package to write the version information, so that is out of the picture anyway. > As said above, if for the reasons you and Henning mention using a package is out of the picture, this is probably the only feasible solution... Except for those people who wrote their own base-files package to resolve this issue, since their package won't work anymore :-) They would have to manage this file using hooks instead of directly declaring the file as managed by their package. This was why I proposed a apt.conf.d hook. Maybe we could generally do it like this. Some package changes /usr/lib/os-release and a apt.conf.d hook would merge this content with ours into /etc/os-release. >> >> The only other way would be to create a new file just for the image or build version, but that would break compatibility now. > Do you mean using a completely different file? Something like /etc/isar-release or /usr/lib/isar-release? If that's what you mean, I wouldn't do it. Not only because of the compatibility issue that you mention, but also because ISAR is de-facto creating a Debian Derivative that information belongs logically into /etc/os-release. Someone finding vanilla Debian inputs in /etc/os-release would expect a vanilla Debian installation, not an ISAR-built distro. While I agree that Isar in some configurations creates a Debian Derivative, currently it puts image version and build date into /etc/os-release. Which is strictly speaking not the right place for that. It would be better to have something like /etc/base-image-version etc. for that. regards, Claudius