From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6522397978051739648 X-Received: by 10.46.44.1 with SMTP id s1mr110007ljs.40.1518690572790; Thu, 15 Feb 2018 02:29:32 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.25.67.66 with SMTP id m2ls2101347lfj.1.gmail; Thu, 15 Feb 2018 02:29:32 -0800 (PST) X-Google-Smtp-Source: AH8x227Dq+moOg0P49Obzzd13BwKI6Wq3IUPnEQcol+e0ITid3wHHaIWgo07VzMRw+W1BEXFVvKk X-Received: by 10.25.205.202 with SMTP id d193mr135356lfg.16.1518690572295; Thu, 15 Feb 2018 02:29:32 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518690572; cv=none; d=google.com; s=arc-20160816; b=layIiY3r+Mv5lXwQbTgiuOYIn6XBF4K7j6RavYu0Z6h0UFWp2hc98BnoOPRLGata6t 1H+Q/OlsdVsYy6w7VJX4lKG7dmg5wUlC9hPJpZMwWPKMnk7JDuld0t8yhWngpoZGvAh3 rJpmIMK32ABhCqmbQvrk2s6wrQsRW7WihI2iIo6QoOVzsvnQUfl34CJiLi9OhleO0Vkc j+aJtxuOKLLFY8NKDZadHEGBnsZCx2bQUV6PGm7kOBUhMXwVld1xdZ9fR2wBfIPo3lRE Xp1apUjr7X6fP7OcEfSrhN3qRYA7GR2zR7cXDbvDmS3kIqzSuy0u9m5KMf3h7GoBfmVm RkuQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject :arc-authentication-results; bh=W0274TcUUH3VduDUkD7ShoUu2SRNMboEz0LR7de1ZH0=; b=VAx+UwbMzaCup+q0WxMz/ArEiPvBWFY/dRHk46utCQOusWMB1+fwjuyMlxOOuMNWLP E5xT8AQcLbkfMVq5S5ewWHyNbglj6rS2i2MhsozrzXHmxZTCrL2HdYAyquQS6ONIX1g+ obdl2wOSpirOP1iX4xDu0kliV+x6PH0l9CQYYGBHWsO1tIK/UneiLr/zS+PapxfJ1k6W +cxQb7P8LMFZ7DBqcI1AA7VQpJnnc5NQGbzR1tGlII7otano8nYe/gWRVR+pklUBjvK3 GpI0OweLoL/gZrFrsfVJTwN39C3MJDbaWd2hqywUsACcYxgjYhEwQEh40VkA84VEpg+o q/TA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id w7si382758ljw.4.2018.02.15.02.29.31 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 15 Feb 2018 02:29:32 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Received: from [10.0.2.15] ([188.227.110.165]) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id w1FATHeh013951 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Thu, 15 Feb 2018 11:29:18 +0100 Subject: Re: [PATCH 2/2] centralize multistrap configuration generation To: Jan Kiszka , isar-users@googlegroups.com, Baurzhan Ismagulov Cc: Claudius Heine References: <20180214131509.16361-1-claudius.heine.ext@siemens.com> <20180214131509.16361-3-claudius.heine.ext@siemens.com> <1eaf1493-e834-ce50-9e75-ff5795348b58@ilbers.de> <923f3293-320b-9172-14aa-3b3a372a1dfa@siemens.com> From: Alexander Smirnov Message-ID: <63ff2a0a-4303-1449-e2e4-41e2049aba7c@ilbers.de> Date: Thu, 15 Feb 2018 13:29:11 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-TUID: zwMoT61cDTcx On 02/15/2018 01:15 PM, Jan Kiszka wrote: > On 2018-02-15 11:05, Claudius Heine wrote: >>>> While this moves us in the right direction, it still leaves some >>>> duplications behind (setup.sh) - and the big question if we should >>>> invest any further in this horrible multistrap mess. >>>> >>>> I've just enabled arm64, and it took a while to figure out that I was >>>> missing some lines in setup.sh. In contrast, qemu-debootstrap just >>>> worked (I used it to debug the multistrap output). >>>> >>>> So, before we move on here: What all does multistrap currently give us, >>>> compared to standard debootstrap? I only see "multiple repositories" so >>>> far. But as Claudius is going to massage that part heavily anyway, I >>>> really wonder if it's worth building on the multistrap indirection any >>>> further. >>>> >>>> How about something like this? >>>> >>>> - [qemu-]debootstrap --variant=minbase ... >>>> - inject all additional apt sources, including user-provided ones >>>> - set pinnings and priority, including user-provided ones >>>> - apt-get dist-upgrade >>>> >>> >>> I'd say that multistrap is a tool which maps user-friendly config file >>> to specific apt repositories configuration. If you look in the >>> do_build() log for buildchroot, you could see what it exactly does: >>> >>> 1. Creating list of sources in rootfs/etc >>> >>> 2. Get packages.gz for all the sources: >>> >>> apt-get  -o Apt::Architecture=armhf -o >>> Dir::Etc::TrustedParts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/trusted.gpg.d >>> -o >>> Dir::Etc::Trusted=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/trusted.gpg.d/trusted.gpg >>> -o Apt::Get::AllowUnauthenticated=true -o Apt::Get::Download-Only=true >>> -o Apt::Install-Recommends=false -o >>> Dir=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/ >>> -o >>> Dir::Etc=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/ >>> -o >>> Dir::Etc::Parts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/apt.conf.d/ >>> -o >>> Dir::Etc::PreferencesParts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/preferences.d/ >>> -o APT::Default-Release=* -o >>> Dir::State=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/lib/apt/ >>> -o >>> Dir::State::Status=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/lib/dpkg/status >>> -o >>> Dir::Cache=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/cache/apt/ >>> update >>> >>> 3. Install packages: >>> >>> apt-get -y  -o Apt::Architecture=armhf -o >>> Dir::Etc::TrustedParts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/trusted.gpg.d >>> -o >>> Dir::Etc::Trusted=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/trusted.gpg.d/trusted.gpg >>> -o Apt::Get::AllowUnauthenticated=true -o Apt::Get::Download-Only=true >>> -o Apt::Install-Recommends=false -o >>> Dir=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/ >>> -o >>> Dir::Etc=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/ >>> -o >>> Dir::Etc::Parts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/apt.conf.d/ >>> -o >>> Dir::Etc::PreferencesParts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/preferences.d/ >>> -o APT::Default-Release=* -o >>> Dir::State=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/lib/apt/ >>> -o >>> Dir::State::Status=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/lib/dpkg/status >>> -o >>> Dir::Cache=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/cache/apt/ >>> install ${LIST_OF_PACKAGES} >>> >>>  > That would kick out all the multistrap complications. >>> >>> For me the plan above intersects with these multistrap steps, you have >>> to re-implement them from the scratch. In this case I'd vote for the >>> community-proved solution. >> >> For me that is a reason to use debootstrap instead of multistrap, since >> multistrap is not community-supported. If we implement some of those >> feature, we might as well do it in debootstrap inself. Just take a look >> at their TODO list: >> >>   Features: >>     ++ second stage via chroot debootstrap/debootstrap >>     ++ debootstrap/deb file to record deb destinations/information >> >>     -- configuration file >>         -- versus command line >>         -- support for sources (vs mirrors) >>         -- faux-pinning for packages >> >> https://anonscm.debian.org/cgit/d-i/debootstrap.git/tree/TODO?id=5d42162bc0fe266b280d81445271e3f56b9560a8 >> >> >> That shows to me that upstream might be interested in some of these >> improvements we would need to implement. > > Exactly. We should really use the community-proven, continuously tested > official approach of Debian, in that in a way that it foresees for it. > > Let's bury multistrap. Its a dead horse we rode to long now. > Nobody is against official tools. :-) This question is open for a quite long time, and the conclusion for now: there is *no* tool that fits Isar requirements for 100%. So in general I only want upstream-first way of tools modification for Isar. Alex Alex