Accessing sbuild build-directory

Accessing sbuild build-directory

[Scheler, Fabian (T CED SES-DE)]

Hi all,

I want to extract the coverage information (i.e. the gcno-files created 
by gcc) that is generated during the package build via sbuild and I look 
for some hook that allows to access the content of the sbuild 
build-directory after the build from within a recipe.

Is this possible?

Ciao
Fabian
-- 
Siemens AG, Technology

Re: Accessing sbuild build-directory

[Anton Mikanovich]

05/04/2024 11:38, 'Scheler, Fabian (T CED SES-DE)' via isar-users wrote:
> Hi all,
>
> I want to extract the coverage information (i.e. the gcno-files 
> created by gcc) that is generated during the package build via sbuild 
> and I look for some hook that allows to access the content of the 
> sbuild build-directory after the build from within a recipe.
>
> Is this possible?
>
> Ciao
> Fabian

Hello Fabian,

You can refer to sbuild documentation for hooks explanation.
The last one executed inside session before its cleanup is
'--chroot-cleanup-commands'.
So you can modify dpkg.bbclass adding any custom commands with this hook.

There are no already implemented dummy pre/post hooks for now, but we 
can try to
implement them in future if needed.

Re: Accessing sbuild build-directory

[Uladzimir Bely]

On Fri, 2024-04-05 at 12:09 +0300, Anton Mikanovich wrote:
> 05/04/2024 11:38, 'Scheler, Fabian (T CED SES-DE)' via isar-users
> wrote:
> > Hi all,
> > 
> > I want to extract the coverage information (i.e. the gcno-files 
> > created by gcc) that is generated during the package build via
> > sbuild 
> > and I look for some hook that allows to access the content of the 
> > sbuild build-directory after the build from within a recipe.
> > 
> > Is this possible?
> > 
> > Ciao
> > Fabian
> 
> Hello Fabian,
> 
> You can refer to sbuild documentation for hooks explanation.
> The last one executed inside session before its cleanup is
> '--chroot-cleanup-commands'.
> So you can modify dpkg.bbclass adding any custom commands with this
> hook.
> 
> There are no already implemented dummy pre/post hooks for now, but we
> can try to
> implement them in future if needed.
> 

FWIW, perhaps 6d5fbbab could be used to quickly play with custom hooks
to sbuild process. A different approach can be chosen for the final
implementation (if any).

-- 
Best regards,
Uladzimir.

Re: Accessing sbuild build-directory

[Scheler, Fabian]

Am 05.04.2024 um 11:57 schrieb Uladzimir Bely:
> On Fri, 2024-04-05 at 12:09 +0300, Anton Mikanovich wrote:
>> 05/04/2024 11:38, 'Scheler, Fabian (T CED SES-DE)' via isar-users
>> wrote:
>>> Hi all,
>>>
>>> I want to extract the coverage information (i.e. the gcno-files
>>> created by gcc) that is generated during the package build via
>>> sbuild
>>> and I look for some hook that allows to access the content of the
>>> sbuild build-directory after the build from within a recipe.
>>>
>>> Is this possible?
>>>
>>> Ciao
>>> Fabian
>>
>> Hello Fabian,
>>
>> You can refer to sbuild documentation for hooks explanation.
>> The last one executed inside session before its cleanup is
>> '--chroot-cleanup-commands'.
>> So you can modify dpkg.bbclass adding any custom commands with this
>> hook.
>>
>> There are no already implemented dummy pre/post hooks for now, but we
>> can try to
>> implement them in future if needed.
>>
> 
> FWIW, perhaps 6d5fbbab could be used to quickly play with custom hooks
> to sbuild process. A different approach can be chosen for the final
> implementation (if any).
> 

thanks for your quick feedback. Yes, injecting arguments to sbuild like 
proposed in 6d5fbbab should be an option. I'll give it a try.

Ciao
Fabian
-- 
Siemens AG, Technology

Re: Accessing sbuild build-directory

[Scheler, Fabian]

Am 05.04.2024 um 12:03 schrieb Scheler, Fabian (T CED EDC-DE):
> Am 05.04.2024 um 11:57 schrieb Uladzimir Bely:
>> On Fri, 2024-04-05 at 12:09 +0300, Anton Mikanovich wrote:
>>> 05/04/2024 11:38, 'Scheler, Fabian (T CED SES-DE)' via isar-users
>>> wrote:
>>>> Hi all,
>>>>
>>>> I want to extract the coverage information (i.e. the gcno-files
>>>> created by gcc) that is generated during the package build via
>>>> sbuild
>>>> and I look for some hook that allows to access the content of the
>>>> sbuild build-directory after the build from within a recipe.
>>>>
>>>> Is this possible?
>>>>
>>>> Ciao
>>>> Fabian
>>>
>>> Hello Fabian,
>>>
>>> You can refer to sbuild documentation for hooks explanation.
>>> The last one executed inside session before its cleanup is
>>> '--chroot-cleanup-commands'.
>>> So you can modify dpkg.bbclass adding any custom commands with this
>>> hook.
>>>
>>> There are no already implemented dummy pre/post hooks for now, but we
>>> can try to
>>> implement them in future if needed.
>>>
>>
>> FWIW, perhaps 6d5fbbab could be used to quickly play with custom hooks
>> to sbuild process. A different approach can be chosen for the final
>> implementation (if any).
>>
> 
> thanks for your quick feedback. Yes, injecting arguments to sbuild like
> proposed in 6d5fbbab should be an option. I'll give it a try.

FYI: Injecting --post-build-commands finally did the trick, but it is 
not really nice. However, there seems not to be another option to 
accomplish that. Unfortunately 6d5fbbab has been reverted and I can 
follow Jan's arguments, maybe this is a use-case justifying the extension.

Ciao
Fabian
-- 
With best regards,
Dr. Fabian Scheler

Siemens AG
T CED EDC-DE
Hertha-Sponer-Weg 3
91058 Erlangen, Germany
Phone: +49 (1522) 1702973
Mobile: +49 (1522) 1702973
mailto:fabian.scheler@siemens.com
www.siemens.com

Siemens Aktiengesellschaft: Chairman of the Supervisory Board: Jim 
Hagemann Snabe; Managing Board: Roland Busch, Chairman, President and 
Chief Executive Officer; Cedrik Neike, Matthias Rebellius, Ralf P. 
Thomas, Judith Wiese; Registered offices: Berlin and Munich, Germany; 
Commercial registries: Berlin-Charlottenburg, HRB 12300, Munich, HRB 
6684; WEEE-Reg.-No. DE 23691322

Re: Accessing sbuild build-directory

[Claudius Heine]

Hi everyone,

On 2024-04-05 3:10 pm, 'Scheler, Fabian' via isar-users wrote:
> Am 05.04.2024 um 12:03 schrieb Scheler, Fabian (T CED EDC-DE):
>> Am 05.04.2024 um 11:57 schrieb Uladzimir Bely:
>>> On Fri, 2024-04-05 at 12:09 +0300, Anton Mikanovich wrote:
>>>> 05/04/2024 11:38, 'Scheler, Fabian (T CED SES-DE)' via isar-users
>>>> wrote:
>>>>> Hi all,
>>>>>
>>>>> I want to extract the coverage information (i.e. the gcno-files
>>>>> created by gcc) that is generated during the package build via
>>>>> sbuild
>>>>> and I look for some hook that allows to access the content of the
>>>>> sbuild build-directory after the build from within a recipe.
>>>>>
>>>>> Is this possible?
>>>>>
>>>>> Ciao
>>>>> Fabian
>>>>
>>>> Hello Fabian,
>>>>
>>>> You can refer to sbuild documentation for hooks explanation.
>>>> The last one executed inside session before its cleanup is
>>>> '--chroot-cleanup-commands'.
>>>> So you can modify dpkg.bbclass adding any custom commands with this
>>>> hook.
>>>>
>>>> There are no already implemented dummy pre/post hooks for now, but we
>>>> can try to
>>>> implement them in future if needed.
>>>>
>>>
>>> FWIW, perhaps 6d5fbbab could be used to quickly play with custom hooks
>>> to sbuild process. A different approach can be chosen for the final
>>> implementation (if any).
>>>
>>
>> thanks for your quick feedback. Yes, injecting arguments to sbuild like
>> proposed in 6d5fbbab should be an option. I'll give it a try.
> 
> FYI: Injecting --post-build-commands finally did the trick, but it is 
> not really nice. However, there seems not to be another option to 
> accomplish that. Unfortunately 6d5fbbab has been reverted and I can 
> follow Jan's arguments, maybe this is a use-case justifying the extension.

I am currently struggling with a similar issue trying to extract the 
`.config` used to compile the swupdate recipe from cip-core [1].

Because the debian package uses multiple different build profiles as 
well as a base debian defconfig to generate a `debian_defconfig`, which 
is then used to generate a `.config` it becomes very difficult to verify 
if the configuration I patched into the base debian defconfig has 
actually applied or was overwritten by something else along the way.

With openembedded you could just look in the build directory and see if 
every option that you wanted is set correctly in the `.config` file, but 
here this is not possible.

It might be possible to add the `.config` into a binary package and read 
it from there, but there should be an option to investigate the build 
directory in isar itself.

[1] 
https://gitlab.com/cip-project/cip-core/isar-cip-core/-/tree/master/recipes-core/swupdate?ref_type=heads
> 
> Ciao
> Fabian

-- 
DENX Software Engineering GmbH,        Managing Director: Erika Unter
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de

Re: Accessing sbuild build-directory

[Jan Kiszka]

On 28.05.24 09:29, Claudius Heine wrote:
> Hi everyone,
> 
> On 2024-04-05 3:10 pm, 'Scheler, Fabian' via isar-users wrote:
>> Am 05.04.2024 um 12:03 schrieb Scheler, Fabian (T CED EDC-DE):
>>> Am 05.04.2024 um 11:57 schrieb Uladzimir Bely:
>>>> On Fri, 2024-04-05 at 12:09 +0300, Anton Mikanovich wrote:
>>>>> 05/04/2024 11:38, 'Scheler, Fabian (T CED SES-DE)' via isar-users
>>>>> wrote:
>>>>>> Hi all,
>>>>>>
>>>>>> I want to extract the coverage information (i.e. the gcno-files
>>>>>> created by gcc) that is generated during the package build via
>>>>>> sbuild
>>>>>> and I look for some hook that allows to access the content of the
>>>>>> sbuild build-directory after the build from within a recipe.
>>>>>>
>>>>>> Is this possible?
>>>>>>
>>>>>> Ciao
>>>>>> Fabian
>>>>>
>>>>> Hello Fabian,
>>>>>
>>>>> You can refer to sbuild documentation for hooks explanation.
>>>>> The last one executed inside session before its cleanup is
>>>>> '--chroot-cleanup-commands'.
>>>>> So you can modify dpkg.bbclass adding any custom commands with this
>>>>> hook.
>>>>>
>>>>> There are no already implemented dummy pre/post hooks for now, but we
>>>>> can try to
>>>>> implement them in future if needed.
>>>>>
>>>>
>>>> FWIW, perhaps 6d5fbbab could be used to quickly play with custom hooks
>>>> to sbuild process. A different approach can be chosen for the final
>>>> implementation (if any).
>>>>
>>>
>>> thanks for your quick feedback. Yes, injecting arguments to sbuild like
>>> proposed in 6d5fbbab should be an option. I'll give it a try.
>>
>> FYI: Injecting --post-build-commands finally did the trick, but it is
>> not really nice. However, there seems not to be another option to
>> accomplish that. Unfortunately 6d5fbbab has been reverted and I can
>> follow Jan's arguments, maybe this is a use-case justifying the
>> extension.
> 
> I am currently struggling with a similar issue trying to extract the
> `.config` used to compile the swupdate recipe from cip-core [1].
> 
> Because the debian package uses multiple different build profiles as
> well as a base debian defconfig to generate a `debian_defconfig`, which
> is then used to generate a `.config` it becomes very difficult to verify
> if the configuration I patched into the base debian defconfig has
> actually applied or was overwritten by something else along the way.
> 
> With openembedded you could just look in the build directory and see if
> every option that you wanted is set correctly in the `.config` file, but
> here this is not possible.
> 
> It might be possible to add the `.config` into a binary package and read
> it from there, but there should be an option to investigate the build
> directory in isar itself.
> 

We should add an option to leave the build artifacts behind when leaving
sbuild, just like we had in the ugly old times with buildchroot. I
wanted to study kernel .config and similar stuff in the past more than
once but always found other ad-hoc workarounds to actually solve that
demand properly.

Jan

-- 
Siemens AG, Technology
Linux Expert Center

Re: Accessing sbuild build-directory

[Claudius Heine]

Hi Jan,

On 2024-05-28 9:58 am, Jan Kiszka wrote:
> On 28.05.24 09:29, Claudius Heine wrote:
>> Hi everyone,
>>
>> On 2024-04-05 3:10 pm, 'Scheler, Fabian' via isar-users wrote:
>>> Am 05.04.2024 um 12:03 schrieb Scheler, Fabian (T CED EDC-DE):
>>>> Am 05.04.2024 um 11:57 schrieb Uladzimir Bely:
>>>>> On Fri, 2024-04-05 at 12:09 +0300, Anton Mikanovich wrote:
>>>>>> 05/04/2024 11:38, 'Scheler, Fabian (T CED SES-DE)' via isar-users
>>>>>> wrote:
>>>>>>> Hi all,
>>>>>>>
>>>>>>> I want to extract the coverage information (i.e. the gcno-files
>>>>>>> created by gcc) that is generated during the package build via
>>>>>>> sbuild
>>>>>>> and I look for some hook that allows to access the content of the
>>>>>>> sbuild build-directory after the build from within a recipe.
>>>>>>>
>>>>>>> Is this possible?
>>>>>>>
>>>>>>> Ciao
>>>>>>> Fabian
>>>>>>
>>>>>> Hello Fabian,
>>>>>>
>>>>>> You can refer to sbuild documentation for hooks explanation.
>>>>>> The last one executed inside session before its cleanup is
>>>>>> '--chroot-cleanup-commands'.
>>>>>> So you can modify dpkg.bbclass adding any custom commands with this
>>>>>> hook.
>>>>>>
>>>>>> There are no already implemented dummy pre/post hooks for now, but we
>>>>>> can try to
>>>>>> implement them in future if needed.
>>>>>>
>>>>>
>>>>> FWIW, perhaps 6d5fbbab could be used to quickly play with custom hooks
>>>>> to sbuild process. A different approach can be chosen for the final
>>>>> implementation (if any).
>>>>>
>>>>
>>>> thanks for your quick feedback. Yes, injecting arguments to sbuild like
>>>> proposed in 6d5fbbab should be an option. I'll give it a try.
>>>
>>> FYI: Injecting --post-build-commands finally did the trick, but it is
>>> not really nice. However, there seems not to be another option to
>>> accomplish that. Unfortunately 6d5fbbab has been reverted and I can
>>> follow Jan's arguments, maybe this is a use-case justifying the
>>> extension.
>>
>> I am currently struggling with a similar issue trying to extract the
>> `.config` used to compile the swupdate recipe from cip-core [1].
>>
>> Because the debian package uses multiple different build profiles as
>> well as a base debian defconfig to generate a `debian_defconfig`, which
>> is then used to generate a `.config` it becomes very difficult to verify
>> if the configuration I patched into the base debian defconfig has
>> actually applied or was overwritten by something else along the way.
>>
>> With openembedded you could just look in the build directory and see if
>> every option that you wanted is set correctly in the `.config` file, but
>> here this is not possible.
>>
>> It might be possible to add the `.config` into a binary package and read
>> it from there, but there should be an option to investigate the build
>> directory in isar itself.
>>
> 
> We should add an option to leave the build artifacts behind when leaving
> sbuild, just like we had in the ugly old times with buildchroot. I
> wanted to study kernel .config and similar stuff in the past more than
> once but always found other ad-hoc workarounds to actually solve that
> demand properly.

So what are the options for implementing this?

It would be good that even after a failed build, it is possible to 
investigate the build directory.

regards,
Claudius

-- 
DENX Software Engineering GmbH,        Managing Director: Erika Unter
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de

Re: Accessing sbuild build-directory

[Jan Kiszka]

On 28.05.24 10:49, Claudius Heine wrote:
> Hi Jan,
> 
> On 2024-05-28 9:58 am, Jan Kiszka wrote:
>> On 28.05.24 09:29, Claudius Heine wrote:
>>> Hi everyone,
>>>
>>> On 2024-04-05 3:10 pm, 'Scheler, Fabian' via isar-users wrote:
>>>> Am 05.04.2024 um 12:03 schrieb Scheler, Fabian (T CED EDC-DE):
>>>>> Am 05.04.2024 um 11:57 schrieb Uladzimir Bely:
>>>>>> On Fri, 2024-04-05 at 12:09 +0300, Anton Mikanovich wrote:
>>>>>>> 05/04/2024 11:38, 'Scheler, Fabian (T CED SES-DE)' via isar-users
>>>>>>> wrote:
>>>>>>>> Hi all,
>>>>>>>>
>>>>>>>> I want to extract the coverage information (i.e. the gcno-files
>>>>>>>> created by gcc) that is generated during the package build via
>>>>>>>> sbuild
>>>>>>>> and I look for some hook that allows to access the content of the
>>>>>>>> sbuild build-directory after the build from within a recipe.
>>>>>>>>
>>>>>>>> Is this possible?
>>>>>>>>
>>>>>>>> Ciao
>>>>>>>> Fabian
>>>>>>>
>>>>>>> Hello Fabian,
>>>>>>>
>>>>>>> You can refer to sbuild documentation for hooks explanation.
>>>>>>> The last one executed inside session before its cleanup is
>>>>>>> '--chroot-cleanup-commands'.
>>>>>>> So you can modify dpkg.bbclass adding any custom commands with this
>>>>>>> hook.
>>>>>>>
>>>>>>> There are no already implemented dummy pre/post hooks for now,
>>>>>>> but we
>>>>>>> can try to
>>>>>>> implement them in future if needed.
>>>>>>>
>>>>>>
>>>>>> FWIW, perhaps 6d5fbbab could be used to quickly play with custom
>>>>>> hooks
>>>>>> to sbuild process. A different approach can be chosen for the final
>>>>>> implementation (if any).
>>>>>>
>>>>>
>>>>> thanks for your quick feedback. Yes, injecting arguments to sbuild
>>>>> like
>>>>> proposed in 6d5fbbab should be an option. I'll give it a try.
>>>>
>>>> FYI: Injecting --post-build-commands finally did the trick, but it is
>>>> not really nice. However, there seems not to be another option to
>>>> accomplish that. Unfortunately 6d5fbbab has been reverted and I can
>>>> follow Jan's arguments, maybe this is a use-case justifying the
>>>> extension.
>>>
>>> I am currently struggling with a similar issue trying to extract the
>>> `.config` used to compile the swupdate recipe from cip-core [1].
>>>
>>> Because the debian package uses multiple different build profiles as
>>> well as a base debian defconfig to generate a `debian_defconfig`, which
>>> is then used to generate a `.config` it becomes very difficult to verify
>>> if the configuration I patched into the base debian defconfig has
>>> actually applied or was overwritten by something else along the way.
>>>
>>> With openembedded you could just look in the build directory and see if
>>> every option that you wanted is set correctly in the `.config` file, but
>>> here this is not possible.
>>>
>>> It might be possible to add the `.config` into a binary package and read
>>> it from there, but there should be an option to investigate the build
>>> directory in isar itself.
>>>
>>
>> We should add an option to leave the build artifacts behind when leaving
>> sbuild, just like we had in the ugly old times with buildchroot. I
>> wanted to study kernel .config and similar stuff in the past more than
>> once but always found other ad-hoc workarounds to actually solve that
>> demand properly.
> 
> So what are the options for implementing this?

man sbuild? I don't known, haven't spent time on investigating it, just
on wanting it.

> 
> It would be good that even after a failed build, it is possible to
> investigate the build directory.
> 

That would be valuable as well, right. It could even become the default
unless there are downsides.

Jan

-- 
Siemens AG, Technology
Linux Expert Center

Re: Accessing sbuild build-directory

[Anton Mikanovich]

28/05/2024 11:52, Jan Kiszka wrote:
> On 28.05.24 10:49, Claudius Heine wrote:
>> Hi Jan,
>>
>> On 2024-05-28 9:58 am, Jan Kiszka wrote:
>>> We should add an option to leave the build artifacts behind when leaving
>>> sbuild, just like we had in the ugly old times with buildchroot. I
>>> wanted to study kernel .config and similar stuff in the past more than
>>> once but always found other ad-hoc workarounds to actually solve that
>>> demand properly.
>> So what are the options for implementing this?
> man sbuild? I don't known, haven't spent time on investigating it, just
> on wanting it.

I was trying --purge* options with sbuild but it didn't work as expected.
Maybe the reason is in schroot managing overlay itself.

As a quick way of handling .config file do_devshell can be used, just 
like the
same is done with kernel in yocto.

Re: Accessing sbuild build-directory

[Jan Kiszka]

On 29.05.24 13:24, Anton Mikanovich wrote:
> 28/05/2024 11:52, Jan Kiszka wrote:
>> On 28.05.24 10:49, Claudius Heine wrote:
>>> Hi Jan,
>>>
>>> On 2024-05-28 9:58 am, Jan Kiszka wrote:
>>>> We should add an option to leave the build artifacts behind when
>>>> leaving
>>>> sbuild, just like we had in the ugly old times with buildchroot. I
>>>> wanted to study kernel .config and similar stuff in the past more than
>>>> once but always found other ad-hoc workarounds to actually solve that
>>>> demand properly.
>>> So what are the options for implementing this?
>> man sbuild? I don't known, haven't spent time on investigating it, just
>> on wanting it.
> 
> I was trying --purge* options with sbuild but it didn't work as expected.
> Maybe the reason is in schroot managing overlay itself.
> 
> As a quick way of handling .config file do_devshell can be used, just
> like the
> same is done with kernel in yocto.
> 

"menuconfig" & Co. is just the tip of the iceberg. To really improve
recipe developer life again, full persistent build output is needed.
Because the .config of the one package is the config.h of the other -
you may imagine.

Jan

-- 
Siemens AG, Technology
Linux Expert Center

Re: Accessing sbuild build-directory

[Baurzhan Ismagulov]

On 2024-05-29 14:33, 'Jan Kiszka' via isar-users wrote:
> > I was trying --purge* options with sbuild but it didn't work as expected.
> > Maybe the reason is in schroot managing overlay itself.
> > 
> > As a quick way of handling .config file do_devshell can be used, just
> > like the same is done with kernel in yocto.
> 
> "menuconfig" & Co. is just the tip of the iceberg. To really improve
> recipe developer life again, full persistent build output is needed.
> Because the .config of the one package is the config.h of the other -
> you may imagine.

To clarify, we agree that a better solution would be handy for development.
We'll be thinking about possible options.

With kind regards,
Baurzhan