From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6761340396568051712 X-Received: by 2002:a63:c805:: with SMTP id z5mr9281384pgg.14.1574338195526; Thu, 21 Nov 2019 04:09:55 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:90a:216c:: with SMTP id a99ls1397662pje.5.gmail; Thu, 21 Nov 2019 04:09:55 -0800 (PST) X-Google-Smtp-Source: APXvYqwtTkZ+a/ZV0lxxg/s/IiqoALWNOcLdK6AeFzn+syIyBxnrIFcmjGANOoyNUED5vK0Vw6dP X-Received: by 2002:a17:90a:353:: with SMTP id 19mr11633336pjf.128.1574338195101; Thu, 21 Nov 2019 04:09:55 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1574338195; cv=none; d=google.com; s=arc-20160816; b=PI4VeelJQwR8TA2vUyk/ZhwX0RtsUM2kBwqA/v7sVr32TmSOInaJMX1Un2KUIhnyst /z/t0NSNIacui7HMOYfj3hjkELZJBnSMx18apvup50aNhSkGTeSely2krBPHlEIY47ca R97M0srkUlztIkQgIBIgtQaCu5JR/p9TEM0DsJgeeSVxjEdi9Vw+/KiEBtmzSVDId+vw aF6GvTwfPTlFtreKmvw388uAHIw9KZuUmj7ajQfKBk7rqWne1SIsxpX4jDay/3oqJL2b 8yfbuxway6W3G1GYQLwrj6n4RvwbRL9HU3bHTUZhZN5vQWsXIv+mhXbb+iM/Mz6WrGfu OjPg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=user-agent:in-reply-to:content-disposition:mime-version:references :message-id:subject:cc:to:date:from:dkim-signature; bh=sl9n2MJ/1KG240mQdpkxkG9ZyMhf5Xd8j95vV7mfBl4=; b=mwBNiZueSE6blB7YBipRYslvxPuAcX/mO95oyZSgBS6FYZ87ds8niuyBob4JAnUUi/ 6R63IGVU7hSuJ3GZFhgbSFWIPVoxUG3aVpV8gD2JZSIvvJxkR6OYwu5dU1JqVLi0IImV NufA/kWWNnf972/TYajltHEQ5x7XYx529u0tk7AwOwKLmXq4ZXebbvV9V85h1acnO1Aw ybpk77MC9Ny9U8ZBhpKKJ48Ov9BwacVA6KkNRa8e/4Yl4ZVgkzOqTWjYOwIgOksL37m7 bm2cr6r+yZwymq4cX/6nOwLe+5bnWaiey6L5XEw+XhFkxBcr0aKdyEvj5HiIyaARkmyd C8CQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=SE4o1kjt; spf=pass (google.com: domain of vijaikumar.kanagarajan@gmail.com designates 2607:f8b0:4864:20::541 as permitted sender) smtp.mailfrom=vijaikumar.kanagarajan@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from mail-pg1-x541.google.com (mail-pg1-x541.google.com. [2607:f8b0:4864:20::541]) by gmr-mx.google.com with ESMTPS id n2si133886pgq.0.2019.11.21.04.09.55 for (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 21 Nov 2019 04:09:55 -0800 (PST) Received-SPF: pass (google.com: domain of vijaikumar.kanagarajan@gmail.com designates 2607:f8b0:4864:20::541 as permitted sender) client-ip=2607:f8b0:4864:20::541; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=SE4o1kjt; spf=pass (google.com: domain of vijaikumar.kanagarajan@gmail.com designates 2607:f8b0:4864:20::541 as permitted sender) smtp.mailfrom=vijaikumar.kanagarajan@gmail.com; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: by mail-pg1-x541.google.com with SMTP id k32so1514516pgl.2 for ; Thu, 21 Nov 2019 04:09:55 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:date:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to:user-agent; bh=sl9n2MJ/1KG240mQdpkxkG9ZyMhf5Xd8j95vV7mfBl4=; b=SE4o1kjtaY+niIH4RqKwR9PEzZY7i4B4MgXSBJhp8Gvt8cxijzWy/8RIyLE9T0Tbip dzFH17TtlNi2K4x5bR0XVhP568pEjs6x4nw5g6xObQKSZVCHTfC1k615J4V8v03aVoEl XGmLiPvErUShzxy8y5FWrX/tdgn7hJJjJLXYsVg+nD+l7num4ntzAqey7yxz1bCW8iC8 ScyUM1Q/frAj4MBNiHPeo/OeF7mTU6nXix4EB4J9Ou/DeysgOin+wmD5crQqWWJbE4YV JrFD0FI3+A/KkzBU17vY6IsKiCzLMiWqoqoTonv17w85OT3xohJJab2BiYAHAzbMX6hF 1BqQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:date:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to:user-agent; bh=sl9n2MJ/1KG240mQdpkxkG9ZyMhf5Xd8j95vV7mfBl4=; b=CPTGwPZ4WbgUofvY3yx2fHHttzRrSmHuoe2kpcISbSeQk2T+fQ2q+Q4xDuqxDpTy75 v5taGDlgGeKjosLoh4A1jh71WNBKzLM1ySOZ9NMfEHHXSRg8LXXeGrc7iD3lzlssDKdl dIIofdo1GiIG35iY0BaToY6OYx8yAX+EXdPPT6PWfOyIMBfUxyspyNvNhhyW/8JLN5fL 1ggaLom67y8b9Q/Fr3ln2x/yExdl8nHqNHJER9hQvBDj8x8abegGUBQjPcltXxTvGQtO pto5kAyzUOZgZjZRLCj6QMQzgd4rYz0SxOYDbMKlJWEnKvJ86fd/V08Xe6jQJt1x6MZg qDxA== X-Gm-Message-State: APjAAAW2WeyT18cO2iUHgxEwCdSgn/6sY3xilHr9QlH2XDZH7h2XRNNh N8hnaE3M1SxyBmJebUH70OuSNQeVdcE= X-Received: by 2002:a63:8e:: with SMTP id 136mr2549616pga.355.1574338194533; Thu, 21 Nov 2019 04:09:54 -0800 (PST) Return-Path: Received: from oxygen (nat-sch.mentorg.com. [139.181.36.34]) by smtp.gmail.com with ESMTPSA id o15sm3204125pgf.2.2019.11.21.04.09.52 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Thu, 21 Nov 2019 04:09:53 -0800 (PST) From: Vijai Kumar K X-Google-Original-From: Vijai Kumar K Date: Thu, 21 Nov 2019 17:39:50 +0530 To: Henning Schild Cc: Vijai Kumar K , isar-users@googlegroups.com Subject: Re: [PATCH] isar-bootstrap: Clear local apt state information Message-ID: <20191121120950.GA23354@oxygen> References: <20191120105340.26844-1-Vijaikumar_Kangarajan@mentor.com> <20191121102218.5458a734@md1za8fc.ad001.siemens.net> <20191121095401.GA15048@oxygen> <20191121124040.2dc3d7a3@md1za8fc.ad001.siemens.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20191121124040.2dc3d7a3@md1za8fc.ad001.siemens.net> User-Agent: Mutt/1.9.4 (2018-02-28) X-TUID: 004bk+6mQf+T On Thu, Nov 21, 2019 at 12:40:40PM +0100, Henning Schild wrote: > Am Thu, 21 Nov 2019 15:24:01 +0530 > schrieb Vijai Kumar K : > > > On Thu, Nov 21, 2019 at 10:22:18AM +0100, Henning Schild wrote: > > > Hi, > > > > > > i am not sure i fully get the problem. With the first stage you > > > probably mean "debootstrap" and the second stage the isar bootstrap > > > on top. > > > > In case of crossdebootstrap, the bootstrapping runs in two stages. > > Basically qemu-debootstrap takes care of that and calls debootstrap > > --second-stage when it identifies the build as crossbuild. > > > > The apt url passed in is not used in the second stage. Instead, it > > uses deb.debian.org/debian and reuses the state information like > > below(log from debootstrap --second-stage) > > > > + mv debootstrap.invalid_dists_stretch_InRelease > > deb.debian.org_debian_dists_stretch_InRelease > > + mv debootstrap.invalid_dists_stretch_Release > > deb.debian.org_debian_dists_stretch_Release > > + mv debootstrap.invalid_dists_stretch_Release.gpg > > deb.debian.org_debian_dists_stretch_Release.gpg > > > > The debootstrap.invalid_* has the InRelease contents of my custom > > repo(a minimal debian repo) passed in as a argument to > > qemu-debootstrap. Post second stage the apt-state information is > > messed up. Has the name of deb.debian.org but contents of my local > > apt mirror. So, if apt-get update is called with this state, it would > > fail when I have both deb.debian.org and the custom repo mentioned in > > my DEBIAN_APT_SOURCES file. > > > > > > > > The first stage should be using the primary entry from the > > > sources.list, so as far as i remember we are just adding more > > > entries. Meaning the state information should stay valid. Maybe in > > > the offline "base-apt" case we have a flow that actually switches > > > the upstream mirror. > > > > > > Maybe you can go into detail how to reproduce the issue at hand, > > > possibly add a test-case for it. > > > > It is one such corner case. Adding a test case for such a unique > > situation is difficult since it involves custom repos. > > I still do not fully get it. > > For the test coverage of custom repos in general, and signed ones on > top, we need to find a solution anyways. > > I would prefer trying to treat the real ones as "custom". i.e. by > removing the default gpg trust from debootstrap and adding it as custom > trust. For this one this trick would not work. > > The second obvious way, which requires some more coding is to use Isar > itself to produce the custom repo. That is most likely the source of > your custom one anyways, it is the source of the one Jan had his custom > gpg keys for. Ah, reading back my comment looks like I am totally against test case for custom repo. I sounded wrong on that. What I was trying to say is we would need to remove /var/lib/apt/lists/* since its a residue left over by qemu-debootstrap. Not removing it has caused me an issue due to the way how debootstrap tool handled the state information. Yes, we should definitely do a test case for custom repo. But I dont think this patch has any effect on that. It is just a proper cleanup before adding our sources list. Thanks, Vijai Kumar K > > Henning > > > > > > > If the files should be deleted, my guess is that the deletion is > > > related to the deletion of sources.list in line 282. So it might be > > > a good idea to add your deletion right after or before that. > > > > Yes. I can move it over there. > > > > > > Thanks, > > Vijai Kumar K > > > > > > > > Henning > > > > > > Am Wed, 20 Nov 2019 16:23:40 +0530 > > > schrieb : > > > > > > > From: Vijai Kumar K > > > > > > > > Debootstrap's second stage switches to using the default debian > > > > mirror. But the apt state information is renamed and used from the > > > > first stage. This might cause apt-get update to fail with the > > > > below error when the sources list contain both custom and > > > > upstream repos. > > > > > > > > | Reading package lists... > > > > | E: Failed to fetch > > > > http://deb.debian.org/debian/dists/stretch/InRelease \ | > > > > Unable to find expected entry 'contrib/source/Sources' in Release > > > > file \ | (Wrong sources.list entry or malformed > > > > file) > > > > > > > > Clear the apt state information before apt-get update. > > > > > > > > Signed-off-by: Vijai Kumar K > > > > --- > > > > meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 1 + > > > > 1 file changed, 1 insertion(+) > > > > > > > > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > > > > b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index > > > > d614d9a..5a379f7 100644 --- > > > > a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++ > > > > b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -320,6 > > > > +320,7 @@ isar_bootstrap() { chroot "${ROOTFSDIR}" /usr/bin/dpkg > > > > --add-architecture ${DISTRO_ARCH} fi > > > > > > > > + rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"* > > > > chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y > > > > chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y -f > > > > chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade > > > > -y \ > > > >