From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6522397978051739648 X-Received: by 10.223.178.178 with SMTP id g47mr483024wrd.12.1518689705487; Thu, 15 Feb 2018 02:15:05 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.28.138.19 with SMTP id m19ls2578624wmd.5.gmail; Thu, 15 Feb 2018 02:15:04 -0800 (PST) X-Google-Smtp-Source: AH8x224UqrKRTKcYPuCkXmu4An8EMAkff4FHwSoPahFi4r+32GALQLF1qWYhZKm1qYfOJyZ7h9Nu X-Received: by 10.28.5.206 with SMTP id 197mr809370wmf.24.1518689704849; Thu, 15 Feb 2018 02:15:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1518689704; cv=none; d=google.com; s=arc-20160816; b=kMqGgqyGRK6YiYUH0TxFuN3OJ37zuDHuTqaZSn5DTqMNoug7iBLFHLjgV9UznBwIj4 3qIo3m/KCT0Cy3Zua1xmSpntblyVd3wkHfPx2Vb3bAEadWYkl2E7kC1wYaRjGP+YAC1e yhCpir57eSRIcKLZixgJBC1AZbG5TJDNv4jUVdO07kOjrUo3h3+h/H0Hs8Ft+95bPmlZ tYYtn1+LzE0QR7VJoKOOclvMMD6Cmg7xBRNjNM2urMzLlrI9CB0AQCHK+3Tz1zlK5aXS M6ofjk/0P7GLWUQF6E8JD+HMc3pVwjjy+xVW8W7frOzVXIaci2FXEODeI9hsC0x6iO8Z +X4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject :arc-authentication-results; bh=0utShLAYgbllgpjfzHacSh3XrWa5J72MyFrZJPkU0xI=; b=FQwD/jSrhirREeCs/5UjC/Bi3uaJznIJGO1CkOpXJIGAcCz1PL4TqMXxIes1kTJQOU 4Y0NpSMrauMuauPS7u/dtAyQn84FzKDeeoVUk/H4KSPXXFwGClZnJPeHGSCYz4QRW3wA WsJOdlX+BAVWNRR2fnGqXbTUqxsIEznqxjTJSr39yL12K+DpOx4jWkiBCb2RKENAuMdO WGqQ7fC6P2I6t9NijiN0PwPUqN6k8+nvpWNllFbPGBRFhzbt75MH63W/zwK1EEXc00WP +lFROZyiWjw767tjhLdUDCfm8Mie1mRDaKLBLK0psGZkpmdVfNc13FSma1t8H+2tfzM4 ZSjQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id c11si928969wrb.0.2018.02.15.02.15.04 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 15 Feb 2018 02:15:04 -0800 (PST) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Received: from mail3.siemens.de (mail3.siemens.de [139.25.208.14]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id w1FAF3ut014994 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 15 Feb 2018 11:15:04 +0100 Received: from [167.87.42.35] ([167.87.42.35]) by mail3.siemens.de (8.15.2/8.15.2) with ESMTP id w1FAF3DI013821; Thu, 15 Feb 2018 11:15:03 +0100 Subject: Re: [PATCH 2/2] centralize multistrap configuration generation To: Claudius Heine , Alexander Smirnov , isar-users@googlegroups.com, Baurzhan Ismagulov Cc: Claudius Heine References: <20180214131509.16361-1-claudius.heine.ext@siemens.com> <20180214131509.16361-3-claudius.heine.ext@siemens.com> <1eaf1493-e834-ce50-9e75-ff5795348b58@ilbers.de> <923f3293-320b-9172-14aa-3b3a372a1dfa@siemens.com> From: Jan Kiszka Message-ID: Date: Thu, 15 Feb 2018 11:15:02 +0100 User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.8.1.12) Gecko/20080226 SUSE/2.0.0.12-1.1 Thunderbird/2.0.0.12 Mnenhy/0.7.5.666 MIME-Version: 1.0 In-Reply-To: <923f3293-320b-9172-14aa-3b3a372a1dfa@siemens.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-TUID: 1oF6jomaJTFt On 2018-02-15 11:05, Claudius Heine wrote: >>> While this moves us in the right direction, it still leaves some >>> duplications behind (setup.sh) - and the big question if we should >>> invest any further in this horrible multistrap mess. >>> >>> I've just enabled arm64, and it took a while to figure out that I was >>> missing some lines in setup.sh. In contrast, qemu-debootstrap just >>> worked (I used it to debug the multistrap output). >>> >>> So, before we move on here: What all does multistrap currently give us, >>> compared to standard debootstrap? I only see "multiple repositories" so >>> far. But as Claudius is going to massage that part heavily anyway, I >>> really wonder if it's worth building on the multistrap indirection any >>> further. >>> >>> How about something like this? >>> >>> - [qemu-]debootstrap --variant=minbase ... >>> - inject all additional apt sources, including user-provided ones >>> - set pinnings and priority, including user-provided ones >>> - apt-get dist-upgrade >>> >> >> I'd say that multistrap is a tool which maps user-friendly config file >> to specific apt repositories configuration. If you look in the >> do_build() log for buildchroot, you could see what it exactly does: >> >> 1. Creating list of sources in rootfs/etc >> >> 2. Get packages.gz for all the sources: >> >> apt-get  -o Apt::Architecture=armhf -o >> Dir::Etc::TrustedParts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/trusted.gpg.d >> -o >> Dir::Etc::Trusted=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/trusted.gpg.d/trusted.gpg >> -o Apt::Get::AllowUnauthenticated=true -o Apt::Get::Download-Only=true >> -o Apt::Install-Recommends=false -o >> Dir=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/ >> -o >> Dir::Etc=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/ >> -o >> Dir::Etc::Parts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/apt.conf.d/ >> -o >> Dir::Etc::PreferencesParts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/preferences.d/ >> -o APT::Default-Release=* -o >> Dir::State=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/lib/apt/ >> -o >> Dir::State::Status=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/lib/dpkg/status >> -o >> Dir::Cache=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/cache/apt/ >> update >> >> 3. Install packages: >> >> apt-get -y  -o Apt::Architecture=armhf -o >> Dir::Etc::TrustedParts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/trusted.gpg.d >> -o >> Dir::Etc::Trusted=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/trusted.gpg.d/trusted.gpg >> -o Apt::Get::AllowUnauthenticated=true -o Apt::Get::Download-Only=true >> -o Apt::Install-Recommends=false -o >> Dir=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/ >> -o >> Dir::Etc=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/ >> -o >> Dir::Etc::Parts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/apt.conf.d/ >> -o >> Dir::Etc::PreferencesParts=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/etc/apt/preferences.d/ >> -o APT::Default-Release=* -o >> Dir::State=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/lib/apt/ >> -o >> Dir::State::Status=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/lib/dpkg/status >> -o >> Dir::Cache=/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/buildchroot/rootfs/var/cache/apt/ >> install ${LIST_OF_PACKAGES} >> >>  > That would kick out all the multistrap complications. >> >> For me the plan above intersects with these multistrap steps, you have >> to re-implement them from the scratch. In this case I'd vote for the >> community-proved solution. > > For me that is a reason to use debootstrap instead of multistrap, since > multistrap is not community-supported. If we implement some of those > feature, we might as well do it in debootstrap inself. Just take a look > at their TODO list: > >   Features: >     ++ second stage via chroot debootstrap/debootstrap >     ++ debootstrap/deb file to record deb destinations/information > >     -- configuration file >         -- versus command line >         -- support for sources (vs mirrors) >         -- faux-pinning for packages > > https://anonscm.debian.org/cgit/d-i/debootstrap.git/tree/TODO?id=5d42162bc0fe266b280d81445271e3f56b9560a8 > > > That shows to me that upstream might be interested in some of these > improvements we would need to implement. Exactly. We should really use the community-proven, continuously tested official approach of Debian, in that in a way that it foresees for it. Let's bury multistrap. Its a dead horse we rode to long now. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux