Re: [PATCH 2/2] centralize multistrap configuration generation

[Jan Kiszka <jan.kiszka@siemens.com>]

On 2018-02-15 11:05, Claudius Heine wrote:
>>> While this moves us in the right direction, it still leaves some
>>> duplications behind (setup.sh) - and the big question if we should
>>> invest any further in this horrible multistrap mess.
>>>
>>> I've just enabled arm64, and it took a while to figure out that I was
>>> missing some lines in setup.sh. In contrast, qemu-debootstrap just
>>> worked (I used it to debug the multistrap output).
>>>
>>> So, before we move on here: What all does multistrap currently give us,
>>> compared to standard debootstrap? I only see "multiple repositories" so
>>> far. But as Claudius is going to massage that part heavily anyway, I
>>> really wonder if it's worth building on the multistrap indirection any
>>> further.
>>>
>>> How about something like this?
>>>
>>> - [qemu-]debootstrap --variant=minbase ...
>>> - inject all additional apt sources, including user-provided ones
>>> - set pinnings and priority, including user-provided ones
>>> - apt-get dist-upgrade
>>>
>>
>> I'd say that multistrap is a tool which maps user-friendly config file
>> to specific apt repositories configuration. If you look in the
>> do_build() log for buildchroot, you could see what it exactly does:
>>
>> 1. Creating list of sources in rootfs/etc
>>
>> 2. Get packages.gz for all the sources:
>>
>> apt-get  -o Apt::Architecture=armhf -o
>> Dir::Etc::TrustedParts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/trusted.gpg.d
>> -o
>> Dir::Etc::Trusted=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/trusted.gpg.d/trusted.gpg
>> -o Apt::Get::AllowUnauthenticated=true -o Apt::Get::Download-Only=true
>> -o Apt::Install-Recommends=false -o
>> Dir=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/
>> -o
>> Dir::Etc=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/
>> -o
>> Dir::Etc::Parts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/apt.conf.d/
>> -o
>> Dir::Etc::PreferencesParts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/preferences.d/
>> -o APT::Default-Release=* -o
>> Dir::State=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/lib/apt/
>> -o
>> Dir::State::Status=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/lib/dpkg/status
>> -o
>> Dir::Cache=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/cache/apt/
>> update
>>
>> 3. Install packages:
>>
>> apt-get -y  -o Apt::Architecture=armhf -o
>> Dir::Etc::TrustedParts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/trusted.gpg.d
>> -o
>> Dir::Etc::Trusted=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/trusted.gpg.d/trusted.gpg
>> -o Apt::Get::AllowUnauthenticated=true -o Apt::Get::Download-Only=true
>> -o Apt::Install-Recommends=false -o
>> Dir=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/
>> -o
>> Dir::Etc=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/
>> -o
>> Dir::Etc::Parts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/apt.conf.d/
>> -o
>> Dir::Etc::PreferencesParts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/preferences.d/
>> -o APT::Default-Release=* -o
>> Dir::State=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/lib/apt/
>> -o
>> Dir::State::Status=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/lib/dpkg/status
>> -o
>> Dir::Cache=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/cache/apt/
>> install ${LIST_OF_PACKAGES}
>>
>>  > That would kick out all the multistrap complications.
>>
>> For me the plan above intersects with these multistrap steps, you have
>> to re-implement them from the scratch. In this case I'd vote for the
>> community-proved solution.
> 
> For me that is a reason to use debootstrap instead of multistrap, since
> multistrap is not community-supported. If we implement some of those
> feature, we might as well do it in debootstrap inself. Just take a look
> at their TODO list:
> 
>   Features:
>     ++ second stage via chroot debootstrap/debootstrap
>     ++ debootstrap/deb file to record deb destinations/information
> 
>     -- configuration file
>         -- versus command line
>         -- support for sources (vs mirrors)
>         -- faux-pinning for packages
> 
> https://anonscm.debian.org/cgit/d-i/debootstrap.git/tree/TODO?id=5d42162bc0fe266b280d81445271e3f56b9560a8
> 
> 
> That shows to me that upstream might be interested in some of these
> improvements we would need to implement.

Exactly. We should really use the community-proven, continuously tested
official approach of Debian, in that in a way that it foresees for it.

Let's bury multistrap. Its a dead horse we rode to long now.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux