Re: image-postproc-extension.bbclass modifying /etc/os-release

[Claudius Heine <ch@denx.de>]

Hi Silvano

On 2021-01-22 10:47, Silvano Cirujano Cuesta wrote:
> Hi Claudius,
> 
> TL;DR: you're proposing here what appears to be a feasible solution to this issue without using a package
> 
> Nevertheless, more information inline.
> 
> On 22/01/2021 09:52, Claudius Heine wrote:
>> Hi Silvano,
>>
>> On 2021-01-19 11:37, Silvano Cirujano Cuesta wrote:
>>> We can create a removal file diversion for /usr/lib/os-release (leaving /etc/os-release untouched, that way the Debian manpage for os-release still applies). This approach has following attributes:
>>>
>>> - /usr/lib/os-release is not managed by any package and that way post-processing it is not an issue and dpkg won't break anything
>>>
>>> - Debian os-release manpage still applies
>>>
>>> If going for the other approach (create our own "base-files" package) we can do pretty much the same:
>>>
>>> - the package doesn't provide /usr/lib/os-release, leaving it package-unmanaged
>>>
>>> - image post-processing to create the file becomes a requirement
>> I looked at the manpage of os-release and it does not state that `/etc/os-release` can or should not be a file:
> 
> That's probably a misunderstanding. I don't mean that we have to create "a file", but provide the os-release information according the specification. There are multiple different valid combinations:
> 
> - /etc/os-release symlink to /usr/lib/os-release
> 
> - /usr/lib/os-relase only (not /etc/os-release at all)
> 
> - /etc/os-release only, no /usr/lib/os-release at all (it's not the recommended way, but it isn't forbidden either)
> 
> - /etc/os-release is a symlink to another file like /usr/lib/isar-release, no /usr/lib/os-release at all (it's not the recommended way, but it isn't forbidden either)

Well I had this in mind:

   - /etc/os-release is a file containing the isar version, 
/usr/lib/os-release is a file containing the original debian version as 
deployed by base-files.

This could also be done similar to your fourth solution:

- /etc/os-release is a symlink to another file like 
/usr/lib/isar-release, original /usr/lib/os-release from base-files

The manual page states that /etc/os-release takes precedence over 
/usr/lib/os-release, so if /etc/os-release exists, it does not matter if 
/usr/lib/os-release does as well or what its content is.

> 
> ...
> 
>>
>> ```
>>         The file /etc/os-release takes precedence over /usr/lib/os-release. Applications should check for the former, and exclusively use its data if it exists, and only fall back to /usr/lib/os-release if
>>         it is missing. Applications should not read data from both files at the same time.  /usr/lib/os-release is the recommended place to store OS release information as part of vendor trees.
>>         /etc/os-release should be a relative symlink to /usr/lib/os-release, to provide compatibility with applications only looking at /etc/. A relative symlink instead of an absolute symlink is necessary
>>         to avoid breaking the link in a chroot or initrd environment such as dracut.
>>
>>         os-release contains data that is defined by the operating system vendor and should generally not be changed by the administrator.
>>
>>         As this file only encodes names and identifiers it should not be localized.
>>
>>         The /etc/os-release and /usr/lib/os-release files might be symlinks to other files, but it is important that the file is available from earliest boot on, and hence must be located on the root file
>>         system.
>> ```
>>
>> So why not make sure `/etc/os-release` is a file and do this:
>>
>>      dpkg-divert --add --local --no-rename /etc/os-release
>>
>> in the post processing step? Shouldn't this prevent any future base-files update from preventing to overwrite the `/etc/os-release`?
> 
> This command would so to say command dpkg not to touch /etc/os-release from the point it's executed on. That way you can have a local version of /etc/os-release and be sure that no package updates is modifying/replacing it. Such a command would go into the postinst hook and another postrm hook would be required to remove the file diversion for /etc/os-release and restoring the original one.

Why would such a command go into a postinst hook? It contains '--local' 
therefore I would expect the administrator doing it him or herself. Or 
in Isars case a bitbake post installation process.

> 
> This would technically resolve the issue, but looks a bit weird to me. Debian and derivatives get that file managed by a package. But since I haven't understood what the issue mentioned by Henning regarding multiconfig and packages, this might be the only feasible solution...

Simple example: Our /etc/os-release contains the build date and git 
commit id. If we put this information into a package, then it we could 
not cache it, because this information will change constantly. That also 
means all dependent tasks and recipes (image creation) cannot be cached, 
breaking the caching of a lot of stuff.

Calling 'bitbake isar-image' two times in short succession without any 
change, would cause 2 full builds of the whole image.

>> The only issue I would see if someone performs a distribution upgrade and now the `/etc/os-release` no longer contains the correct version, but other than an apt.conf hook that updates `/etc/os-release` based on new information from `/usr/lib/os-release` after an upgrade, I don't see a way to do this automatically.
> 
> Why should a distribution upgrade change that? I'm not aware that dist-upgrade changes file diversions... To my knowledge, once the file diversion has been created, dpkg should completely ignore that file.
> 
> File diversions don't apply to packages, but to files, therefore I'd expect it to survive a dist-upgrade. Therefore dpkg doesn't keep track of the packages that are "blocked" from touching a file. It keeps track of the file path and of the package that requested the file diversion (unless "--local" specified, as you propose).

You misunderstood me.

In my example a base-files package would still update 
/usr/lib/os-release to contain a new debian suite name, if you do a 
distribution update. If you want our diverted /etc/os-release to contain 
this new info automatically, you could do this via a apt.conf.d hook.

> 
>>
>> I would rather divert '/etc/os-release' in a postprocessing script than `/usr/lib/os-release` because, I see the post-processing more of a local configuration.
> 
> Diverting /etc/os-release or /usr/lib/os-release depends on what we want to achieve (it would be even better letting the ISAR user decide it):

I don't want to divert /usr/lib/os-release. Let it contain the upstream 
version for reference. This file does not matter, because 
/etc/os-release takes precedence over it.

> 
> 1. Keep originating Debian for reference? Then change only /etc/os-release.

Yes.

> 
> 2. Wipe away all information about originating Debian? Then change /usr/lib/os-release.

No.

> 
> /etc/os-release might appear like a local configuration for the ISAR builder, but IMO it's intrinsic information about the distribution being built with ISAR. As the documentation that you are quoting says "defined by the operating system vendor and should generally not be changed by the administrator." Therefore IMO the path /etc/os-release was kept for historical reasons (see footnote 1 of the announcement [1]), but it's an exception to the convention that files under "/etc" are configurations and can be changed.

Well its a separate discussion if Isar is in the role of a distribution 
vendor or of an administrator.

I would say its something in between.

> 
> [1] http://0pointer.de/blog/projects/os-release
> 
>> For multiple reasons we cannot used a package to write the version information, so that is out of the picture anyway.
> As said above, if for the reasons you and Henning mention using a package is out of the picture, this is probably the only feasible solution... Except for those people who wrote their own base-files package to resolve this issue, since their package won't work anymore :-) They would have to manage this file using hooks instead of directly declaring the file as managed by their package.

This was why I proposed a apt.conf.d hook. Maybe we could generally do 
it like this.

Some package changes /usr/lib/os-release and a apt.conf.d hook would 
merge this content with ours into /etc/os-release.

>>
>> The only other way would be to create a new file just for the image or build version, but that would break compatibility now.
> Do you mean using a completely different file? Something like /etc/isar-release or /usr/lib/isar-release? If that's what you mean, I wouldn't do it. Not only because of the compatibility issue that you mention, but also because ISAR is de-facto creating a Debian Derivative that information belongs logically into /etc/os-release. Someone finding vanilla Debian inputs in /etc/os-release would expect a vanilla Debian installation, not an ISAR-built distro.

While I agree that Isar in some configurations creates a Debian 
Derivative, currently it puts image version and build date into 
/etc/os-release. Which is strictly speaking not the right place for that.

It would be better to have something like /etc/base-image-version etc. 
for that.

regards,
Claudius