Re: RFC: need to support package builds ala pbuilder?

[Jan Kiszka <jan.kiszka@siemens.com>]

On 13.09.18 15:05, Hombourger, Cedric wrote:
> Hello all,
> 
> I recently came across an interesting case that may require us providing a mechanism to support building packages in their own private buildchroot
> Let me first describe the issue:
> 
> # Isar defines two tasks to build Debian packages: (1) do_prepare and (2) do_build
> # The former installs build dependencies while the latter does the actual build
> #
> # The Isar lock is acquired for do_prepare_build to serialize access to the package
> # database. While this looks ok, we may have builds fail in the following scenario
> #
> #     core 1                           core 2
> #     -------------------------------- ---------------------------
> #
> #     recipe1:do_prepare_build
> #      |  download dependencies
> #      |  install libssl-dev
> #      |  task completes
> #                                       recipe2:do_prepare_build
> #     recipe2:do_build                   | download dependencies
> #      | autoconf                        | remove libssl-dev
> #      | make                            | install libssl1.0-dev
> #
> # Running "autoconf" or "make" while libssl-dev gets removed to allow installation of
> # libssl1.0-dev may cause either to fail since OpenSSL headers / libraries will be
> # temporarily removed

If recipe2 (or did you rather meant recipe3?) depends on libssl-dev, and that is
built without a proper dependency encoded, that's a recipe bug. that build step
must not start prio to the deploy_deb step of the producing recipe is done.

> 
> To keep locking simple and avoid introducing a big fat lock for the entire package build
> (do_prepare_build + do_build), adding an option for a recipe to instruct bitbake to create
> a buildchroot for a specific package (e.g. tmp/work/my-distro-amd64/my-package/buildchroot)
> may be necessary. This would result in something conceptually similar to pbuilder but
> integrated into Isar. The downside for those packages is that we would likely end-up
> downloading the same packages again (not a big deal for those of you using a local
> caching proxy such as apt-cacher-ng)
> 
> I therefore wanted to seek your opinion in either the need for such a mechanism or alternate solutions you may have in mind?

The above thing aside, having "clean-room" package built rather than sharing the
buildchroot would be a valuable feature, to catch recipe bugs early and more
reliably.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux