Re: [PATCH 0/3] use local repo for multistrap and drop "dpkg -i"

[Henning Schild <henning.schild@siemens.com>]

Am Thu, 10 Aug 2017 14:46:27 +0200
schrieb Baurzhan Ismagulov <ibr@radix50.net>:

> Hello Henning,
> 
> On Tue, Aug 01, 2017 at 05:24:05PM +0200, Henning Schild wrote:
> > This series introduces support for a local repository to be used
> > instead of "dpkg -i" after bootstrap.  
> 
> Before we discuss it in detail: We have an implementation of apt
> generation that is also used for binary package caching across
> bitbake runs in the lenormf/develop-l20170602-dpkg-cross branch
> provided to Hans and mentioned on the list. Which advantages does
> your implementation bring?

As i said before there are too many branches and forks around. I would
like to keep the discussion on master and patches under review. A
comparison makes sense after someone sketched the changes in a few
sentences.

> I like your dpkg-scanpackages approach compared to reprepro in the
> existing implementation. Analyzing the rest and choosing / combining
> the optimal strategy will take some time. 

I am not sure i like it because it is a plain directory. Reprepro
probably has its advantages but is more complicated.

> To speed up the process,
> could you perhaps sketch the changes in the workflow in a few
> sentences?

The idea behind it is to introduce one policy. Everything that goes
into the rootfs comes out of debian packages and their hooks.
Multistrap is the only entrance to the rootfs.

- solves dependency tracking compared to "dpdk -i"
- limits fetching of debs to multistrap
 - if you fetch with "apt-get" in the chroot you get proxy problems
   again
 - you loose track of what you used for the rootfs
 - your rootfs needs apt-get installed
- makes multistrap the valid candidate to tell us exactly what we used
  and allow us to create a partial mirror for reproducible builds
 - evaluate "retainsources" feature

Changes in the workflow:
- create a repo as the first image-task
 - this one has to wait for all .debs to be built and copied
- drop do_populate which messes with the rootfs after our new
  gatekeeper multistrap

> Regarding the kernel: Which arch and suite are you building for? We
> don't install kernels in QEMU images because they are not necessary.

That was just me building incorrectly, i did not know about the
"multiconfig:" thing.
 
> Regarding the increase of build time, I'd really like to avoid that.

Me too, that is why i started the discussion on caching. Giving up on
the gatekeeper idea would IMHO not be the way to go.

> Building all configurations in one run already takes up to 40 min.

CI is patient and for basic testing one does not need to build all of
them.

> The problem should be easily mitigatable by splitting rootfs
> multistrap and apt-getting the built packages into separate recipes.

Yes but that is what i mean with "giving up the gatekeeper". We could
implement both ways controlled by a variable in a config. That would
speed up the builds during development and allow switching to strict
gatekeeper mode for releases.

Henning

> At least I'm not aware of any increase in the existing implementation.