From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6631131420328722432 X-Received: by 2002:a50:8eca:: with SMTP id x10mr4900720edx.8.1544029786367; Wed, 05 Dec 2018 09:09:46 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:906:3484:: with SMTP id g4-v6ls4385885ejb.8.gmail; Wed, 05 Dec 2018 09:09:45 -0800 (PST) X-Google-Smtp-Source: AFSGD/WMb5n2AyJ2zIZVnxJalzwrCB6mDiMgCg2tb/vAcHldlPqz1rKLfw1VVu+bGStef7SN9f8D X-Received: by 2002:a17:906:6899:: with SMTP id n25-v6mr4132363ejr.11.1544029785968; Wed, 05 Dec 2018 09:09:45 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1544029785; cv=none; d=google.com; s=arc-20160816; b=F16KyHxjVCaVcoHx0XBFnXkDvCl0PcoCEljjHD+hgmHja5ddjVEyySpXxoDTCUQNG3 fJrBydEkp68MiFA7j1DaKU0m7/S3K1dq4fAIoZd7hDApc0XBPbYbeZYXmkvFFoDb44+b zPgulan4T3P8nfXWJAmTo8vPFNC4XfegnzpdpV8dnyhj1UkAVrQJ8+sFezDkzM5nicDK MFortryP2u0G28bkTXf3LdXT6hjt7r2PSMDEXMoplP6ZjXWbBuk1OZrsz6WuzV+ZVpYT pxMayjywjiWot6y/hClxHxcK7XOxvQ7cvpRgPj+tjqCkfoiAH7jBm26REVlqWK4Du+Nq 05dg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=c3qUYyvuJAcBhPCPvBW4mLse2K2RsaQ3NMoFst0oZ0g=; b=phqF6pQI+Ly6+lS/+3C4B1dtK3rrOCbgheEm8VYS8lYYojXiQ2j5MVtwDQdl6E1fZz W6higaMi4eiOAnmpoK5+nEUyqlh2m06SHRK3qGlSIFgAk4l3MOXdiavMfOhBvAtVM5Is XgxeHgAYpiScpHLTX4QVC42d28zMf8gUfzCSu0NVQxLTR2Pxq53SS5ry56xgVZmZHSMv 01ncF/D7QxUvwvA8Gi5WCnq7KVjusaIIMtNN1S9BinT1pX5cPoyzLSiP1RcE5K1Ctw1K C648nyY0pEwYi/s2M3HDtV7KDbt9tCRymL29K69jdPst7xREgxtInumY7L4NB1gAhUJ4 iX7g== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Return-Path: Received: from goliath.siemens.de (goliath.siemens.de. [192.35.17.28]) by gmr-mx.google.com with ESMTPS id m24si986337edc.3.2018.12.05.09.09.45 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 05 Dec 2018 09:09:45 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) client-ip=192.35.17.28; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id wB5H9i0i012676 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Wed, 5 Dec 2018 18:09:44 +0100 Received: from md1za8fc.ad001.siemens.net ([139.25.69.6]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id wB5H9iAI001942; Wed, 5 Dec 2018 18:09:44 +0100 Date: Wed, 5 Dec 2018 18:09:43 +0100 From: Henning Schild To: "[ext] Claudius Heine" Cc: Tolga =?UTF-8?B?SG/Fn2fDtnI=?= , , Subject: Re: [PATCH] isar-bootstrap: debootstrap --include flag to support https:// sources Message-ID: <20181205180943.7a19e523@md1za8fc.ad001.siemens.net> In-Reply-To: References: <20181204133544.50621-1-tolga.hosgor@siemens.com> <45323f2d-7dac-9b9d-6382-013b9ae1242c@siemens.com> X-Mailer: Claws Mail 3.15.0-dirty (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: quoted-printable X-TUID: rTD8mgSnClwA Am Wed, 5 Dec 2018 13:27:34 +0100 schrieb "[ext] Claudius Heine" : > Hi Tolga, >=20 > On 05/12/2018 11.28, Tolga Ho=C5=9Fg=C3=B6r wrote: > > On Tue, Dec 4, 2018 at 5:46 PM Claudius Heine > > wrote: =20 > >> > >> Hi Jan, > >> > >> On 04/12/2018 15.28, Jan Kiszka wrote: =20 > >>> On 04.12.18 15:23, [ext] Claudius Heine wrote: =20 > >>>>> @@ -138,6 +154,12 @@ def get_distro_components_argument(d, > >>>>> is_host): else: > >>>>> return "" > >>>>> +def get_debootstrap_includes(d, is_host=3DFalse): > >>>>> + if get_distro_have_https_source(d, is_host): > >>>>> + return "locales,apt-transport-https,ca-certificates" =20 > >>>> > >>>> Those package names should be configured in the distro config. =20 > >=20 > > I partly agree. It makes sense that it is the responsibility of > > distro config but "debootstrap" does this automatically [1]. This > > intervention by "debootstrap" creates some erratic behavior. =20 >=20 > debootstrap has subscripts for each distribution, most of them link > to the same script, but that doesn't change the fact that they > already have a mechanism in place to implement different logic > depending on the distribution. >=20 > isar-bootstrap has to work with every possible debootstrap=20 > implementation, so in order to do that, it shouldn't contain distro=20 > specific stuff. >=20 > >=20 > > Scenario 1: > > - No special bootstrap config on distro. > > - Distro sources have an "https" and "http" in that order. > >=20 > > Scenario 2: > > - No special bootstrap config on distro. > > - Distro sources have an "http" and "https" in that order. > >=20 > > While we were happily building on scenario 1, with a minor change > > now the build magically fails on scenario 2. > >=20 > > An acceptable solution for consistency would be to disable this > > behavior [1] but "debootstrap" does not seem to have an option to > > disable this magic-package-adding behavior. How about explicitly > > `--exclude` the packages for consistency? =20 >=20 > No explicit excluding of packages please. That would hard code our=20 > implementation to internal debootstrap stuff. But if we just add the=20 > required packages to debootstrap to handle https when we detect the=20 > requirement, then this should work in what ever order the repository=20 > sources come. >=20 > > "debootstrap" also did this in its distro-specific scripts [1], like > > your approach. But now it is on a common script "debian-common" [2]. > > So now they think this is more or less a standard package I guess. =20 >=20 > debootstrap should do what ever debootstrap does. What it does in > detail shouldn't concern isar. (at least as long as everything works) >=20 > > Actually what is needed is something the output of: > > "DEBOOTSTRAP_DIR=3D/usr/share/debootstrap source > > /usr/share/debootstrap/scripts/; MIRRORS=3D > > work_out_debs && echo $base" but this is not publicly exposed in any > > way. Could call `debootstrap --print-debs ` for each mirror > > and sort/uniq the result but it would be ugly. =20 >=20 > Don't go that way. That will just lead us implementing our own=20 > debootstrap and that is not currently the scope of isar. >=20 > >>> Means you would suggest to introduce two variables, e.g. > >>> DEBOOTSTRAP_INCLUDE and DEBOOTSTRAP_INCLUDE_HTTPS to encode those > >>> packages? Those could then be set in distro confs and simply used > >>> here. =20 > >> =20 > >=20 > > Now I am completely lost. I suppose there is no need to create the > > logic to to differentiate HTTP and HTTPS in ISAR if distro conf will > > set the packages. Distro configuration knows whether if it is using > > HTTP or HTTPS (or in theory something else) and it should be enough > > to define/extend _one_ variable accordingly. =20 >=20 > I would suggest calling those variables like this: >=20 > DISTRO_BOOTSTRAP_BASE_PACKAGES =3D "locales" > DISTRO_BOOTSTRAP_BASE_PACKAGES_append_https-support =3D "=20 > apt-transport-https ca-certificates" I like that idea. Because what Isar asks for is "https-support" and it should not directly carry what that means ... in terms of packages to install. Looking at the debootstrap scripts we will end up with the two packages mentioned already ... for all distros. But we are much cleaner and ready for another distro to rename those packages. Henning =20 > # OVERRIDE =3D "...:https-support:..." # when you detect that https is > needed. >=20 > or similar. And then use those variables in isar-bootstrap with your > logic. >=20 > > Do you agree on adding > > `--exclude=3Dapt-transport-https,ca-certificates` and introducing a > > `BOOTSTRAP_PACKAGES` variable to be configured in distro? > >=20 > > [1] > > https://salsa.debian.org/installer-team/debootstrap/blob/08cfced48bdf7b= c1240efb63f69cde7ed385038b/scripts/sid#L36-40 > > [2] > > https://salsa.debian.org/installer-team/debootstrap/blob/4a4e7186de7237= c58eae4f0b2de3ad670ad7436f/scripts/debian-common#L30-34 =20 >=20 > NACK for exclude > ACK for BOOTSTRAP_PACKAGES variables (or similar named, see above) >=20 > kind regards, > Claudius >=20 >=20