Re: [PATCH 2/2] centralize multistrap configuration generation

[Alexander Smirnov <asmirnov@ilbers.de>]

On 02/15/2018 01:15 PM, Jan Kiszka wrote:
> On 2018-02-15 11:05, Claudius Heine wrote:
>>>> While this moves us in the right direction, it still leaves some
>>>> duplications behind (setup.sh) - and the big question if we should
>>>> invest any further in this horrible multistrap mess.
>>>>
>>>> I've just enabled arm64, and it took a while to figure out that I was
>>>> missing some lines in setup.sh. In contrast, qemu-debootstrap just
>>>> worked (I used it to debug the multistrap output).
>>>>
>>>> So, before we move on here: What all does multistrap currently give us,
>>>> compared to standard debootstrap? I only see "multiple repositories" so
>>>> far. But as Claudius is going to massage that part heavily anyway, I
>>>> really wonder if it's worth building on the multistrap indirection any
>>>> further.
>>>>
>>>> How about something like this?
>>>>
>>>> - [qemu-]debootstrap --variant=minbase ...
>>>> - inject all additional apt sources, including user-provided ones
>>>> - set pinnings and priority, including user-provided ones
>>>> - apt-get dist-upgrade
>>>>
>>>
>>> I'd say that multistrap is a tool which maps user-friendly config file
>>> to specific apt repositories configuration. If you look in the
>>> do_build() log for buildchroot, you could see what it exactly does:
>>>
>>> 1. Creating list of sources in rootfs/etc
>>>
>>> 2. Get packages.gz for all the sources:
>>>
>>> apt-get  -o Apt::Architecture=armhf -o
>>> Dir::Etc::TrustedParts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/trusted.gpg.d
>>> -o
>>> Dir::Etc::Trusted=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/trusted.gpg.d/trusted.gpg
>>> -o Apt::Get::AllowUnauthenticated=true -o Apt::Get::Download-Only=true
>>> -o Apt::Install-Recommends=false -o
>>> Dir=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/
>>> -o
>>> Dir::Etc=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/
>>> -o
>>> Dir::Etc::Parts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/apt.conf.d/
>>> -o
>>> Dir::Etc::PreferencesParts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/preferences.d/
>>> -o APT::Default-Release=* -o
>>> Dir::State=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/lib/apt/
>>> -o
>>> Dir::State::Status=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/lib/dpkg/status
>>> -o
>>> Dir::Cache=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/cache/apt/
>>> update
>>>
>>> 3. Install packages:
>>>
>>> apt-get -y  -o Apt::Architecture=armhf -o
>>> Dir::Etc::TrustedParts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/trusted.gpg.d
>>> -o
>>> Dir::Etc::Trusted=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/trusted.gpg.d/trusted.gpg
>>> -o Apt::Get::AllowUnauthenticated=true -o Apt::Get::Download-Only=true
>>> -o Apt::Install-Recommends=false -o
>>> Dir=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/
>>> -o
>>> Dir::Etc=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/
>>> -o
>>> Dir::Etc::Parts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/apt.conf.d/
>>> -o
>>> Dir::Etc::PreferencesParts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/preferences.d/
>>> -o APT::Default-Release=* -o
>>> Dir::State=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/lib/apt/
>>> -o
>>> Dir::State::Status=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/lib/dpkg/status
>>> -o
>>> Dir::Cache=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/cache/apt/
>>> install ${LIST_OF_PACKAGES}
>>>
>>>   > That would kick out all the multistrap complications.
>>>
>>> For me the plan above intersects with these multistrap steps, you have
>>> to re-implement them from the scratch. In this case I'd vote for the
>>> community-proved solution.
>>
>> For me that is a reason to use debootstrap instead of multistrap, since
>> multistrap is not community-supported. If we implement some of those
>> feature, we might as well do it in debootstrap inself. Just take a look
>> at their TODO list:
>>
>>    Features:
>>      ++ second stage via chroot debootstrap/debootstrap
>>      ++ debootstrap/deb file to record deb destinations/information
>>
>>      -- configuration file
>>          -- versus command line
>>          -- support for sources (vs mirrors)
>>          -- faux-pinning for packages
>>
>> https://anonscm.debian.org/cgit/d-i/debootstrap.git/tree/TODO?id=5d42162bc0fe266b280d81445271e3f56b9560a8
>>
>>
>> That shows to me that upstream might be interested in some of these
>> improvements we would need to implement.
> 
> Exactly. We should really use the community-proven, continuously tested
> official approach of Debian, in that in a way that it foresees for it.
> 
> Let's bury multistrap. Its a dead horse we rode to long now.
> 

Nobody is against official tools. :-) This question is open for a quite 
long time, and the conclusion for now: there is *no* tool that fits Isar 
requirements for 100%. So in general I only want upstream-first way of 
tools modification for Isar.

Alex

Alex