Re: sbuild migration problem: how to jump into SBUILD_CHROOT with build dependencies installed

[Florian Bezdeka <florian.bezdeka@siemens.com>]

On 19.08.22 12:32, Baurzhan Ismagulov wrote:
> Hello Florian,
> 
> On Fri, Aug 19, 2022 at 10:36:52AM +0200, Florian Bezdeka wrote:
>>>> dpkg_runbuild_prepend() {
>>>>      sudo chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \
>>>>          sh -c "autoreconf -fi"
>>>> }
>>>>
>>>> In the pre-sbuild times this task was executed *after* the build
>>>> dependencies have been installed but *before* the dpkg_runbild task.
>>>>
>>>> With sbuild:
>>>> - There is no "install build dependencies" task anymore
>>>>    Installation of build dependencies is done by sbuild internally
>>>> - The task is now executed *before* build dependencies are installed
>>>> - I still could chroot into ${SBUILD_CHROOT}, but the build dependencies
>>>>    will not be there
>>>>
>>>> Is there any possibility to jump into the ${SBUILD_CHROOT} after build
>>>> dependencies are installed but before the actual packet build?
>>>
>>> Correct way to implement this is to move autoreconf into debian/rules.
>>
>> In this simple example for sure. But: There might be recipes where
>> debian/rules is not directly "accessable", forcing me to wait for 3rd
>> parties to update their build process (or patch things in my layer) and
>> all of this just because I tried to update Isar...
>>
>> If there is no way to jump into ${SBUILD_CHROOT} with build dependencies
>> installed we have a (very minor) regression.
> 
> I'd like to spend a few words on the background: The sbuild change is not about
> the build tool per se; it's rather about re-using the Debian package building
> interface. In Debian, a developer prepares a standalone dsc file, uploads it to
> a build server, and buildd produces binary packages in a clean-room
> environment. No intermediate steps are taken in between; all necessary
> information is contained in the source package and its dependencies.
> 
> Clean-room building is a requirement because otherwise binaries could be e.g.
> linked with libraries which were not intended by the developer. There are also
> architectural reasons like modifiability, etc. Yes, removing interfaces is
> technically a regression but the resulting functionality is more important.
> That is why we'd like to follow the Debian way and avoid custom steps. If this
> is not obviously applicable, we want to see the use cases and discuss them in
> detail.

That's all fine. Clean-room building is definitely the way to go.

> 
> Regarding autoreconf, rebuilding the files can be done in debian/rules (there
> is also dh_autoreconf). That said, I personally prefer to generate the files
> during the development phase, commit them to the VCS, and disable maintainer
> mode for building. This saves time, reduces dependencies, and avoids a number
> of issues.
> 

autoreconf was just a simple example. There is more in the wild which
can not be moved into debian/rules easily. Most of the time it's not
really a technical issue. See below.

> Regarding the debian/rules being inaccessible, in that case I don't understand
> yet how you build it with Isar in the first place. I suggest that you share
> your source package (possibly offline) and we look what could be done. The idea
> is that any extra steps go either into preparing the source package, or into
> package patches -- so yes, patching may be required.


Sorry, I can't easily share that. But: We have a group of Isar users
doing integration tasks. They're fetching sources from different teams /
organization units and building product images out of them.

If the integrator is unable to update build tools (Isar is one of them)
because of a broken build, they're stuck. Long term: Try pushing all the
3rd parties into the right direction, short term: Do not update Isar or
start patching.

Pushing them into the right direction normally takes months... Patching
is a nightmare. Skipping updates is easy and wins.

So whenever we break Isar APIs, we must be careful. Keeping a bitbake
task but changing the conditions when the task is invoked is dangerous
and frustrates users.

Conclusion: Jumping into ${SBUILD_CHROOT} with build dependencies
already installed is no longer possible.

Thanks for your comments!

Let's hope I can somehow solve that issue ;-)

> 
> We've seen a number of such source packages in e.g. meta-iot2050 and it does
> require some work if the upstream follows the "let's fetch 128 deps from
> everywhere and bootstrap ourselves" process. In many cases, you want to have
> clean source packages yourself to comply with license requirements w.r.t.
> rebuilding. It is also a big help for the users of the package, since anyone
> can then apt-get source pkg; sudo apt-get build-dep pkg; cd pkg-ver;
> dpkg-buildpackage -uc -us for any given package. So, custom package recipes
> should ideally just build the source package and leave building to Debian
> tools.
> 
> With kind regards,
> Baurzhan
>