Re: [RFC] using lightweight containers instead of chroot

[Jan Kiszka <jan.kiszka@siemens.com>]

On 12.07.21 12:54, Helmut Grohne wrote:
> Hi Jan,
> 
> On Mon, Jul 12, 2021 at 10:25:41AM +0200, Jan Kiszka wrote:
>> On 08.07.21 21:34, Helmut Grohne wrote:
>>> The reason to want DPKG_ROOT is to prepare the root filesystem for your
>>> embedded board. If that's not what you want to do, DPKG_ROOT is not your
>>> tool.
>>
>> We want it for both the board and the build env. We have the same issues
>> to solve there, means installing packages in unprivileged manner, either
>> for target arch or the builder arch (cross and native builds need to be
>> supported). So I see DPKG_ROOT as a building block for both. In
>> addition, it would overcome qemu-user, which is speed gain.
> 
> No. Let us for a moment assume that DPKG_ROOT would just work for all
> packages in Debian (and that is a far stretch really). Then you'd be
> able to install your Build-Depends into a directory without using
> chroot. But how do you build your package then? You need to chroot the
> dpkg-buildpackage call. So you are back to requiring chroot, at which
> point you can just install your whole build environment using chroot.
> 
> DPKG_ROOT is the wrong tool for that job. The one thing that it fixes is
> requiring qemu for the embedded filesystem image. Cross builds already
> work entirely without qemu (including package installation).
> 

We can't cross-build all packages, unfortunately. That's why we will
continue to require native installations for some packages'
buildchroots, just like we obviously do for the target rootfs. If you
can create and also augment (install deps) those without qemu, you gain
speed. Granted, those continue to require qemu for the actual build.
Which brings me back to still having to add namespace support to
binfmt_misc...

>> Yes, we prepare (ad-hoc debianization) or patch or just use
>> debian/control for that. Packages should then be taken from the
>> configured deb repos, including local isar-apt, and using a pre-defined
>> apt database (only updated once so that all builds use the same package
>> versions). Local caching of the fetched deb packages is used in
>> addition, both as optimization as well as a building block for offline
>> builds.
> 
> I'm sorry, but none of this explains why you need to patch a package
> after it is unpacked.
> 
> Before building starts, you can just download all the packages that are
> to be patched, patch them and include the patched ones in isar-apt. At
> that point, you can run without hooks.

Which implies fetching and unpacking them first. Yes, that might happen
in Isar without bothering the builder, and that might be a way to not
having to intercept its regular run. We don't do that right now, we just
kick off dpkg-buildpackage on the unpacked sources.

We also do that so far for packages that don't come as source packages
in the first place but as debianized repos or as ad-hoc debianized
sources (deb_debianize or local debian/ folder coming from an Isar
layer). Those would require an upfront source package creation as well,
something that dpkg-buildpackage currently does while building. Possibly
cleaner.

> 
>> One purpose of Isar is ad-hoc fix-ups for upstream Debian packages. E.g.
>> to inject patches before they are accepted upstream or to address issues
>> that are not compatible with upstream packaging. Recent example:
>> https://github.com/siemens/meta-iot2050/commit/a1d35d5a3494236b2cdb0bd310c3ff0d4600d955
>> (some patches not even accepted upstream yet but needed).
> 
> I do see why you need to patch packages. I do not see why that needs to
> happen inside the build. Why not do it before building?

As long as it happens in the same Isar build run, that is fine, I agree.

> 
>> This procedure should be automated and expressed with very few
>> statements. The current approaches we have are far from optimal, but the
>> vision remains that you just tell Isar the package, the patch, and then
>> it does whatever is best from Debian perspective. That may include the
>> mentioned source package patching upfront. That will have to go into our
>> recipes and classes.
> 
> That sounds a lot like your requirement is a consequence of your current
> implementation and not an actual requirement. I fully agree that such
> patching needs to be simple to perform and (unfortunately) is a frequent
> thing to do. Just the how we do it is sub-optimal both in Isar and
> rebootstrap.

We are not bound to the current implementation, just the "user
interface" should not be needlessly changed. Means that the user
provides source URIs, artifacts etc. via bitbake recipes and gets a
patched and (re-)built package when building the related target.

> 
>> Sure. We are not building hundreds of packages, but it also not only
>> about building one or two. The other reason is the mentioned identical
>> versions in all buildchroots. Reproducing them must provide identical
>> content (where common) for a single image build run.
> 
> I think I already debunked the performance aspect. For reproducibility,
> you do want a clean build to reproduce the artifacts of a previous one.
> Reproducibility needs to work regardless of whether you cache the base
> image. Indeed, the reproducible folks are working on reproducible
> installations somewhat and the dpkg-root-demo explicitly verifies
> reproducibility (given the same inputs, but you do cache them anyway for
> offline builds).
> 
> So again, the requirement of caching an image seems to be a consequence
> of your current implementation.

In parts but not generally, as you pointed out yourself ("given the same
inputs").

> 
>> Currently, we have no secure container env. The container simply serves
>> as deployment tool of Isar's dependencies and builder-side
>> configurations so that you can run it on any distro, not only Debian.
>> And you can run it in CI identically to your host.
> 
> If your docker container is not setup securely, you can make it support
> user namespaces and be done. :)

You seem to confuse containers and docker. Our vision is unprivileged
podman (or also docker-whatever, if that is still around by then) being
able to build complete images. Still a long way to go, though.

> 
>> Isar users use Isar, they usually do not modify the core. And the goal
>> of Isar is to avoid that they have to but rather provide easy means to
>> configure things that must be configurable. If the builder would fall
>> under this, e.g. no one build would build all types of packages (hope
>> that is not the case, so far it wasn't), it would become an Isar
>> requirement to switching them without much effort.
> 
> How would I disagree with that? Of course providing users with the
> flexibility to switch their builder turns out to become a requirement
> once users rely on it. However, the "without much effort" part is the
> thing that mdbp is supposed to provide. In theory, your builder becomes
> a shell command with a sane default and your user can change that one
> command to switch the builder, because the API differences are hidden
> behind a common mdbp API.
> 
>> That is a valid point we need to keep an eye on. If mdbp solved that
>> already, it would be one of the added values I mentioned that could make
>> it relevant for Isar, even if not requiring to switch builders.
> 
> Great. Given that builder-flexibility seems to be a non-requirement for
> you, I suggest that you use sbuild and look into the mdbp-sbuild
> implementation to figure what kind of settings you need to fix to make
> it reproducible. That's also a form of using it. ;)
> 

Good! :)

Thanks,
Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux