RE: [meta-eid] Re: multistrap support

[<kazuhiro3.hayashi@toshiba.co.jp>]

Hi Claudius, (and Isar developers)

Thank you for your replies,
I would like to reply one by one.

> -----Original Message-----
> From: meta-eid@googlegroups.com [mailto:meta-eid@googlegroups.com] On
> Behalf Of Claudius Heine
> Sent: Wednesday, February 13, 2019 10:56 PM
> To: hayashi kazuhiro(林 和宏 ○ＳＷＣ□ＯＳＴ)
> <kazuhiro3.hayashi@toshiba.co.jp>; isar-users@googlegroups.com
> Cc: meta-eid@googlegroups.com
> Subject: [meta-eid] Re: multistrap support
> 
> Hi Kazu,
> 
> I don't speak for the whole Isar community, but just from my point of
> view. For context I implemented the switch from multistrap to debootstrap.
> 
> On 13/02/2019 14.14, kazuhiro3.hayashi@toshiba.co.jp wrote:
> > Hi,
> >
> > I would like to know if isar still has some plans to use multistrap again
> > (or similar existing tools if exists) to generate target images.
> 
> No, only debootstrap is supported and we will most probably stick with that.
> 
> > Multistrap for isar-image-base has been removed by the following commit:
> >
> https://github.com/ilbers/isar/commit/19a314559178f7afd93ce3dafe8c8647
> ca6c8884
> > and replaced by setup_root_file_system() in
> isar-bootstrap-helper.bbclass,
> > which seems to use (pre-built?)
> isar-bootstrap-$ROOTFS_DISTRO-$ROOTFS_ARCH
> > as the base tree instead of running the debootstrap process.
> >
> > I guess that the main purposes of quitting multistrap is that
> > there is no big update for a few years (though the last update is 2.2.10
> on Nov. 2018)
> > Are there any other reasons?
> 
> That is one of them. Debootstrap is better supported by the main Debian
> community than any other bootstrapping tools. And since it is out of
> scope or the Isar Project to implement its own bootstrapping mechanism,
> we will stick with the second most official one 'debootstrap'. The
> Debian installer being the most official way to install Debian, which is
> a bit to bulky and integrated to be easily used in a scripted environment.

I'm fully agree that using debootstrap, one of the core package in Debian
for a long time, as the base tool to generate images from multiple apt repos
if we decided not to use multistrap.
Regarding Debian installer, base-installer(udeb) also uses debootstrap
to install the base system before using apt and tasksel, but as you said,
it's integrated to d-i scripts so difficult to use in non-installer environment.

> 
> Both multistrap and debootstrap had or still have some inconveniences
> with our usage and it would of course be great to have one that fits
> exactly our purposes and is used and supported by the broader
> Debian/Embedded Debian and possible Debian server community. But as I
> said Isar does not have the resources to implement that.

Thank you for your opinion.
I didn't mean that multistrap should be used in Isar again.
Also, my opinion is that any other full 'replacement' of debootstrap
should not be implemented if bootstrapping from multiple repositories
can be achieved by pure debootstrap + additional scripts.

(Sorry, I was misunderstanding multistrap; I thought it uses
 debootstrap as back-end, but that is a replacement actually.)

The current bitbake function in Isar is one of the better solutions.
I should check more detailed implementation of that.

> 
> > (Some bugs difficult to be fixed, mismatches with isar specification,
> etc.)
> >
> > The current isar-bootstrap based approach would work fine for isar system,
> > but in that case, it might be difficult to share efforts for
> > developing and maintaining the functionality with non isar users.
> > I'm just interested in the future plan of isar.
> 
> Maybe you can give an example about which feature you think could be
> moved outside of the isar project and be used separately.

I should learn the mechanism of setup_root_file_system() more,
but at least I'm originally considering how I can provide the following API:

mybootstrap [debootstrap_OPTION] <suite> <target> [<mirror_list> [<script>]]

I guess that the most important part for me is isar_bootstrap() in isar-bootstrap.inc,
which is shared by isar-bootstrap-*.bb and copied by setup_root_file_system as
a base tree ("${DEPLOY_DIR_IMAGE}/isar-bootstrap-$ROOTFS_DISTRO-$ROOTFS_ARCH/).
Other remaining commands in setup_root_file_system() seems to be almost based on apt,
so too easy to control multiple repositories.
In the following commands, only one debootstrap command is called with
one repository argument given from get_distro_primary_source_entry().
https://github.com/ilbers/isar/blob/master/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc#L175
Does it work with multiple apt repositories?
Or, installing required packages from other repos with apt after debootstrapping finishes?

> 
> One issue here is that Isar uses bitbake as the build task scheduler, so
> shell scripts etc. are difficult to extract from it to be used outside
> of Isar. And if they are extracted from the Isar recipes, then using
> them in Isar again is disadvantageous for the Isar build process.
> 
> So sharing code between a bitbake/isar project and other build systems
> is not really possible.

I usually have the same issues in my bitbake recipes development.
It might be possible to share the concrete processes to create
a Debian base system from multiple repositories,
even if it's difficult to share the actual codes.

Background of my question:
I guess that people who are developing Debian based system
with their own custom packages (I'm one of them) may want to use
a command (or a wrapper script) to create a base image from multiple apt repos,
especially if they need to replace a part of existing Debian essential packages
by their custom packages (coreutils, util-linux, dpkg, etc.).
I'm investigating existing approaches that provide the API above or similar one.

Regards,
Kazu