Re: [PATCH v7 00/10] Improving base-apt usage

[Uladzimir Bely <ubely@ilbers.de>]

On Thu, 2024-08-01 at 08:57 +0000, MOESSBAUER, Felix wrote:
> On Thu, 2024-07-25 at 18:07 +0300, Uladzimir Bely wrote:
> > `base-apt` is a local apt repository containing all upstream
> > (Debian,
> > Raspberry Pi OS, Ubuntu...) packages needed for a particular build.
> > This series implements upfront repository downloading. This is the
> > first step towards local partial mirror management.
> > 
> > The current approach in `next`:
> > 
> > - On the first build, debootstrap and sbuild are used for building
> > Isar artifacts. The packages downloaded from the Internet are
> > cached
> > in local directories.
> > - On the next build:
> >   - Analyze the logs from the previous build, save packages
> > downloaded by the bootstraps, sbuilds and imagers into `base-apt`.
> >   - Use `base-apt` for bootstrapping, building and image creation.
> > 
> > Some issues with the current approach:
> > 
> > 1. Different policies must be followed for the first and the
> > subsequent builds.
> > 2. As we have multiple versions of the same package from the main
> > and
> > security repositories and rely on build logs and `find` for
> > populating `base-apt`, extra care must be taken to ensure that the
> > right package version lands in `base-apt`.
> > 3. We rely on internal implementation of `debootstrap` and `sbuild`
> > for saving and reusing the packages. Changing to e.g. `mmdebstrap`
> > breaks the unrelated `base-apt` functionality.
> 
> Hi, what is the overall plan w.r.t. this patch series? My
> understanding
> was that we want to switch to mmdebstrap rather sooner than later.
> But
> now you write that this will be incompatible with mmdebstrap.
> 

Hello.

"Rely on internal implementation" means that we currently workaround
some things related to deb-dl-import / deb-dl-export :
 - sbuild: 2-stage deb cache import/export: (downloads/deb => import to
workdir/rootfs => symlinking to apt cache in sbuild chroot => build =>
export to workdir/rootfs => export to downloads/deb):
 - mmdebstrap: since it is not possible to mmdebstrap into non-empty
dir, we can't simply use deb-dl-import before mmdebstrap called and
have to use some hooks to workaround it.

With new base-apt approach, deb-dl import/export (and all related
workarounds) becomes redundant. When new approach becomes stable enough
we could completely remove fallback mode with deb-dl import/export
functionality.

> For me, this series still looks pretty experimental.

That's why currently we saved fallback mode when things should work
like before (ISAR_PREFETCH_BASE_APT = "0")

> 
> Felix
> 
> > 4. Source packages are stored in a different flat directory, `apt-
> > get
> > source` for upstream packages is not possible.
> > 5. At the moment of `base-apt` creation all we have is the package
> > name. The knowledge about the upstream repositories is lost and no
> > local repository policy implementation is possible (e.g., for the
> > "multiple products, multiple distros" use case).
> > 6. For implementing further use cases like "fetch all sources
> > necessary for bootstrapping the base system itself", additional
> > logic
> > is necessary.
> > 
> 
> -- 
> Siemens AG, Technology
> Linux Expert Center
> 
> 

-- 
Best regards,
Uladzimir.



-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/8a9f43a239129e12f09ccfeda8b683b9f36a8fc2.camel%40ilbers.de.