From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6761340396568051712 X-Received: by 2002:adf:fc0a:: with SMTP id i10mr10514809wrr.105.1574336442842; Thu, 21 Nov 2019 03:40:42 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a1c:1b0b:: with SMTP id b11ls3697166wmb.3.gmail; Thu, 21 Nov 2019 03:40:42 -0800 (PST) X-Google-Smtp-Source: APXvYqyo9z39JFaVfzLbemHlG/fXfG/YL6qFbZJ/0dsiUeLJfsnDGpG8k5gPVccl2Yegx2mBchg8 X-Received: by 2002:a7b:c5d0:: with SMTP id n16mr5523543wmk.78.1574336442154; Thu, 21 Nov 2019 03:40:42 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1574336442; cv=none; d=google.com; s=arc-20160816; b=pEX2PLvPBZcBueAdk8xO/KnN5J/Xihl2Uk5H2p+dbhmI32JRARMrSvQzEeU5zzBpXy /3iFmMtfuMTjP+MDfc0HphlAV6DGk4wCgP9BqEFcF0gh8P0altB1lcTBJmkB3B7jZQlR j8eczUD1kYtx34IDn+QBtQcy/ibx2nM6MFI8BG83HKl+2tlkyelDTmI7lWmqxFl766PP YSTx89p7urESl8PsO+yrP9Vgx9ev61huiV9uXvOOhQIWCQNvNTf6SpGMcJSIHtd1RpOV /p1TGmT815j4KxGlreIlFNCHWrQVuOvVL/FNWZQ17FwbJETucvAAh5KI36DqRI1DxQjU HJKg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=t7IwtNMzoxhelagkGnEfxjyMS0A/pVwOnxtRlOWJeLo=; b=dgb91y21fRITavvJe8X1I+Ju648qEpDGrqL+RiBLGYJTSSD3d3Al1VGjPlqp7f1Xuk Y2Xwp2szlXy7j+6jbTXaJjH2Ruf5MMOAODwuOA/kzf1bjdl60NBBIj+hevDbhshWi6BL 0uwNq21KnBf4iLUjeYFnHJeY6l2OgM13Ap0ChPq+Gfl3U89zs73yUIHYew5pd/sp5aNz 5Bb9Qr+2bois4CZNGw9pAyoppunAnkmYUm3QI9xtqBBTwurQqe0Yhmwg2eQUWoe1jWnU G/OsPS76eqoicXkGVvf672J7KfFZZ1yID61/86Y/6JVcsF7SXDk73DQVM0qTxGOW8Qz9 gSKQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from gecko.sbs.de (gecko.sbs.de. [194.138.37.40]) by gmr-mx.google.com with ESMTPS id b198si423914wmd.4.2019.11.21.03.40.42 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 21 Nov 2019 03:40:42 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.40 as permitted sender) client-ip=194.138.37.40; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by gecko.sbs.de (8.15.2/8.15.2) with ESMTPS id xALBef62008623 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 21 Nov 2019 12:40:41 +0100 Received: from md1za8fc.ad001.siemens.net ([139.25.0.65]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id xALBefVX006875; Thu, 21 Nov 2019 12:40:41 +0100 Date: Thu, 21 Nov 2019 12:40:40 +0100 From: Henning Schild To: Vijai Kumar K Cc: Subject: Re: [PATCH] isar-bootstrap: Clear local apt state information Message-ID: <20191121124040.2dc3d7a3@md1za8fc.ad001.siemens.net> In-Reply-To: <20191121095401.GA15048@oxygen> References: <20191120105340.26844-1-Vijaikumar_Kangarajan@mentor.com> <20191121102218.5458a734@md1za8fc.ad001.siemens.net> <20191121095401.GA15048@oxygen> X-Mailer: Claws Mail 3.17.4 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: 4aOsknwcncoD Am Thu, 21 Nov 2019 15:24:01 +0530 schrieb Vijai Kumar K : > On Thu, Nov 21, 2019 at 10:22:18AM +0100, Henning Schild wrote: > > Hi, > > > > i am not sure i fully get the problem. With the first stage you > > probably mean "debootstrap" and the second stage the isar bootstrap > > on top. > > In case of crossdebootstrap, the bootstrapping runs in two stages. > Basically qemu-debootstrap takes care of that and calls debootstrap > --second-stage when it identifies the build as crossbuild. > > The apt url passed in is not used in the second stage. Instead, it > uses deb.debian.org/debian and reuses the state information like > below(log from debootstrap --second-stage) > > + mv debootstrap.invalid_dists_stretch_InRelease > deb.debian.org_debian_dists_stretch_InRelease > + mv debootstrap.invalid_dists_stretch_Release > deb.debian.org_debian_dists_stretch_Release > + mv debootstrap.invalid_dists_stretch_Release.gpg > deb.debian.org_debian_dists_stretch_Release.gpg > > The debootstrap.invalid_* has the InRelease contents of my custom > repo(a minimal debian repo) passed in as a argument to > qemu-debootstrap. Post second stage the apt-state information is > messed up. Has the name of deb.debian.org but contents of my local > apt mirror. So, if apt-get update is called with this state, it would > fail when I have both deb.debian.org and the custom repo mentioned in > my DEBIAN_APT_SOURCES file. > > > > > The first stage should be using the primary entry from the > > sources.list, so as far as i remember we are just adding more > > entries. Meaning the state information should stay valid. Maybe in > > the offline "base-apt" case we have a flow that actually switches > > the upstream mirror. > > > > Maybe you can go into detail how to reproduce the issue at hand, > > possibly add a test-case for it. > > It is one such corner case. Adding a test case for such a unique > situation is difficult since it involves custom repos. I still do not fully get it. For the test coverage of custom repos in general, and signed ones on top, we need to find a solution anyways. I would prefer trying to treat the real ones as "custom". i.e. by removing the default gpg trust from debootstrap and adding it as custom trust. For this one this trick would not work. The second obvious way, which requires some more coding is to use Isar itself to produce the custom repo. That is most likely the source of your custom one anyways, it is the source of the one Jan had his custom gpg keys for. Henning > > > > If the files should be deleted, my guess is that the deletion is > > related to the deletion of sources.list in line 282. So it might be > > a good idea to add your deletion right after or before that. > > Yes. I can move it over there. > > > Thanks, > Vijai Kumar K > > > > > Henning > > > > Am Wed, 20 Nov 2019 16:23:40 +0530 > > schrieb : > > > > > From: Vijai Kumar K > > > > > > Debootstrap's second stage switches to using the default debian > > > mirror. But the apt state information is renamed and used from the > > > first stage. This might cause apt-get update to fail with the > > > below error when the sources list contain both custom and > > > upstream repos. > > > > > > | Reading package lists... > > > | E: Failed to fetch > > > http://deb.debian.org/debian/dists/stretch/InRelease \ | > > > Unable to find expected entry 'contrib/source/Sources' in Release > > > file \ | (Wrong sources.list entry or malformed > > > file) > > > > > > Clear the apt state information before apt-get update. > > > > > > Signed-off-by: Vijai Kumar K > > > --- > > > meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 1 + > > > 1 file changed, 1 insertion(+) > > > > > > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > > > b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index > > > d614d9a..5a379f7 100644 --- > > > a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ > > > b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -320,6 > > > +320,7 @@ isar_bootstrap() { chroot "${ROOTFSDIR}" /usr/bin/dpkg > > > --add-architecture ${DISTRO_ARCH} fi > > > > > > + rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"* > > > chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y > > > chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y -f > > > chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade > > > -y \ > >