Re: isar-bootstrap

[Jan Kiszka <jan.kiszka@siemens.com>]

On 26.10.21 23:15, Henning Schild wrote:
> Am Tue, 26 Oct 2021 22:48:53 +0200 (CEST)
> schrieb ydirson@free.fr:
> 
>>> For the download Isar goes the pragmatic way and lets debian fetch
>>> what
>>> it wants. With a few exceptions ... i.e. there is only one "global
>>> apt-get update" so you have to hope that you can apt-get install
>>> what that initial run created your external database for. In
>>> practice that does not fail too ofter ... or you have to clean
>>> build again.
>>>
>>> If you really need to pin debian down to what it fetches, because
>>> for some reason (like repro build) you need your own mirror. In
>>> fact Isar spits out a partial debian mirror after an "online" build
>>> (base-apt). That can be used for consecutive offline builds, or as
>>> a base for consecutive "online" builds with custom
>>> DISTRO_APT_SOURCES.
>>>
>>> While snapshots.debian.org can be used as DISTRO_APT_SOURCES mirror
>>> in
>>> theory ... in practice it has rate-limiting in place. So you might
>>> succeed in a manual build that you retry over and over (or a small
>>> image), but in CI without caching ... you will never get a big image
>>> to
>>> build. That rate-limiting issue will need to be discussed with
>>> snapshots, we are not the first ones to have issues with it.
>>> But i personally would tell people to simply not freeze if they can,
>>> and the ones that need to freeze i would in fact tell to get a full
>>> debian mirror of their own, instead of a partial one produced by
>>> isars
>>> base-apt.
>>> As an OSS project you might see less of a need of freezing, tracking
>>> in
>>> fact is a security feature ... and debian will not do much more than
>>> security on their stable distros.  
>>
>> This is more of a concern for reproducibility of the build process
>> at package level.  Probably this was not published very widely, but
>> there has been work on Debian package reproducibility in the context
>> of Qubes already, including a solution to the snapshots.d.o problem:
>>
>> https://forum.qubes-os.org/t/reproducible-builds-for-debian-a-big-step-forward/6800
> 
> Cool stuff! I will give that service a try and look into where the tool
> debrebuild comes into play. A tool that might in itself add some value.
> 
> But to me it all seems like a workaround of the rate-limiting on
> snapshots. A problem that can maybe be solved with funding, in case it
> is a cost-problem. In Siemens people end up with their own
> implementations of snapshots, which in sum likely costs more than
> funding the real one for public non-limited use (together with other
> stakeholders).
> Still getting funding right and stable might be harder than any costly
> workaround downstream.
> I think i also heard trust arguments against snapshots, but those might
> just be pointless. Even if it is not-Debian (is it?) there will be gpg
> in place, but then you still need to trust that you get what you asked
> for because if a snapshot holds back a security update gpg would not be
> able to tell.
> 
> But yes, good stuff! I bet that can be used in Isar somehow, but maybe
> only in your layers. You might not want the load from others on your
> infra. And the amd64 limitation is kind of severe. We have arm64
> becoming more and more relevant and even riscv working for early
> prototypes. And i686 and arm32 floating around in places for some time
> to be maintained.

Missing non-amd64 support is exactly the reason why we are not yet
broadly using that alternative snapshot - too many of our use cases are
armhf or arm64 (extremely few are still i386). Compared to that, [1] is
then only a minor issue.

Reproducibility is a strategic topic for us, just not yet one with
highest urgency. However, people start to look into this in Isar context
already (see e.g. [2]).

Jan

[1] https://github.com/fepitre/debian-snapshot/issues/2
[2] https://groups.google.com/g/isar-users/c/GXYNmEpn-ns/m/tb7MKx3FAAAJ

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux