Re: [PATCH] isar-bootstrap: Clear local apt state information

[Henning Schild <henning.schild@siemens.com>]

Am Thu, 21 Nov 2019 15:24:01 +0530
schrieb Vijai Kumar K <vijaikumar.kanagarajan@gmail.com>:

> On Thu, Nov 21, 2019 at 10:22:18AM +0100, Henning Schild wrote:
> > Hi,
> > 
> > i am not sure i fully get the problem. With the first stage you
> > probably mean "debootstrap" and the second stage the isar bootstrap
> > on top.  
> 
> In case of crossdebootstrap, the bootstrapping runs in two stages.
> Basically qemu-debootstrap takes care of that and calls debootstrap
> --second-stage when it identifies the build as crossbuild.
> 
> The apt url passed in is not used in the second stage. Instead, it
> uses deb.debian.org/debian and reuses the state information like
> below(log from debootstrap --second-stage)
> 
> + mv debootstrap.invalid_dists_stretch_InRelease
> deb.debian.org_debian_dists_stretch_InRelease
> + mv debootstrap.invalid_dists_stretch_Release
> deb.debian.org_debian_dists_stretch_Release
> + mv debootstrap.invalid_dists_stretch_Release.gpg
> deb.debian.org_debian_dists_stretch_Release.gpg
> 
> The debootstrap.invalid_* has the InRelease contents of my custom
> repo(a minimal debian repo) passed in as a argument to
> qemu-debootstrap. Post second stage the apt-state information is
> messed up. Has the name of deb.debian.org but contents of my local
> apt mirror. So, if apt-get update is called with this state, it would
> fail when I have both deb.debian.org and the custom repo mentioned in
> my DEBIAN_APT_SOURCES file.
> 
> > 
> > The first stage should be using the primary entry from the
> > sources.list, so as far as i remember we are just adding more
> > entries. Meaning the state information should stay valid. Maybe in
> > the offline "base-apt" case we have a flow that actually switches
> > the upstream mirror.
> > 
> > Maybe you can go into detail how to reproduce the issue at hand,
> > possibly add a test-case for it.  
> 
> It is one such corner case. Adding a test case for such a unique
> situation is difficult since it involves custom repos.

I still do not fully get it.

For the test coverage of custom repos in general, and signed ones on
top, we need to find a solution anyways.

I would prefer trying to treat the real ones as "custom". i.e. by
removing the default gpg trust from debootstrap and adding it as custom
trust. For this one this trick would not work.

The second obvious way, which requires some more coding is to use Isar
itself to produce the custom repo. That is most likely the source of
your custom one anyways, it is the source of the one Jan had his custom
gpg keys for.

Henning

> > 
> > If the files should be deleted, my guess is that the deletion is
> > related to the deletion of sources.list in line 282. So it might be
> > a good idea to add your deletion right after or before that.  
> 
> Yes. I can move it over there.
> 
> 
> Thanks,
> Vijai Kumar K
> 
> > 
> > Henning
> > 
> > Am Wed, 20 Nov 2019 16:23:40 +0530
> > schrieb <vijaikumar.kanagarajan@gmail.com>:
> >   
> > > From: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
> > > 
> > > Debootstrap's second stage switches to using the default debian
> > > mirror. But the apt state information is renamed and used from the
> > > first stage. This might cause apt-get update to fail with the
> > > below error when the sources list contain both custom and
> > > upstream repos.
> > > 
> > > | Reading package lists...
> > > | E: Failed to fetch
> > > http://deb.debian.org/debian/dists/stretch/InRelease \ |
> > > Unable to find expected entry 'contrib/source/Sources' in Release
> > > file \ |		(Wrong sources.list entry or malformed
> > > file)
> > > 
> > > Clear the apt state information before apt-get update.
> > > 
> > > Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
> > > ---
> > >  meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 1 +
> > >  1 file changed, 1 insertion(+)
> > > 
> > > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> > > b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc index
> > > d614d9a..5a379f7 100644 ---
> > > a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc +++
> > > b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc @@ -320,6
> > > +320,7 @@ isar_bootstrap() { chroot "${ROOTFSDIR}" /usr/bin/dpkg
> > > --add-architecture ${DISTRO_ARCH} fi
> > >  
> > > +            rm -rf "${ROOTFSDIR}/var/lib/apt/lists/"*
> > >              chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y
> > >              chroot "${ROOTFSDIR}" /usr/bin/apt-get install -y -f
> > >              chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade
> > > -y \  
> >