[PATCH 1/1] Revert "buildchroot: Do not build cross when there are no arch-specific outputs"

[PATCH 1/1] Revert "buildchroot: Do not build cross when there are no arch-specific outputs"

[Felix Moessbauer]

This reverts commit 563986703d9a0405c70af9b50ecedede2ac59cbd.

The reverted patch made a shortcut to simplify the handling of cross-build
dependencies of architecture:all packages.
This is not valid for the following reasons:

1. Just scanning the control file for 'all' is not sufficient as
a source package might generate both arch specific and :all packages.

2. This breaks architecture specific transitive build dependencies.
These mainly apply to correctly packaged python packages that support
cross building and use setuptools. By that, the transitive dependency
libpython3.9-minimal:<target-arch> is not attracted but instead the version
for the host, leading to very hard to debug build-time issues.

In case packages do not handle their cross-build dependencies correctly, we
should not try to work around in ISAR, but better enforce upstream patches.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 meta/recipes-devtools/buildchroot/files/deps.sh | 6 ------
 1 file changed, 6 deletions(-)

diff --git a/meta/recipes-devtools/buildchroot/files/deps.sh b/meta/recipes-devtools/buildchroot/files/deps.sh
index 1d617bc8..ccfc460c 100644
--- a/meta/recipes-devtools/buildchroot/files/deps.sh
+++ b/meta/recipes-devtools/buildchroot/files/deps.sh
@@ -27,12 +27,6 @@ if [ "$3" != "--download-only" ]; then
         -o APT::Get::List-Cleanup="0"
 fi
 
-# Do not set an architecture when building only 'all' (generic) packages.
-# This can avoid unneeded cross-build issues.
-if ! grep "^Architecture:" debian/control | grep -qv "all"; then
-    set_arch=""
-fi
-
 control_file=$(pwd)/debian/control
 cd ..
 
-- 
2.30.2

Re: [PATCH 1/1] Revert "buildchroot: Do not build cross when there are no arch-specific outputs"

[Jan Kiszka]

On 22.12.21 13:55, Felix Moessbauer wrote:
> This reverts commit 563986703d9a0405c70af9b50ecedede2ac59cbd.
> 
> The reverted patch made a shortcut to simplify the handling of cross-build
> dependencies of architecture:all packages.
> This is not valid for the following reasons:
> 
> 1. Just scanning the control file for 'all' is not sufficient as
> a source package might generate both arch specific and :all packages.
> 
> 2. This breaks architecture specific transitive build dependencies.
> These mainly apply to correctly packaged python packages that support
> cross building and use setuptools. By that, the transitive dependency
> libpython3.9-minimal:<target-arch> is not attracted but instead the version
> for the host, leading to very hard to debug build-time issues.
> 
> In case packages do not handle their cross-build dependencies correctly, we
> should not try to work around in ISAR, but better enforce upstream patches.
> 
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
>  meta/recipes-devtools/buildchroot/files/deps.sh | 6 ------
>  1 file changed, 6 deletions(-)
> 
> diff --git a/meta/recipes-devtools/buildchroot/files/deps.sh b/meta/recipes-devtools/buildchroot/files/deps.sh
> index 1d617bc8..ccfc460c 100644
> --- a/meta/recipes-devtools/buildchroot/files/deps.sh
> +++ b/meta/recipes-devtools/buildchroot/files/deps.sh
> @@ -27,12 +27,6 @@ if [ "$3" != "--download-only" ]; then
>          -o APT::Get::List-Cleanup="0"
>  fi
>  
> -# Do not set an architecture when building only 'all' (generic) packages.
> -# This can avoid unneeded cross-build issues.
> -if ! grep "^Architecture:" debian/control | grep -qv "all"; then
> -    set_arch=""
> -fi
> -
>  control_file=$(pwd)/debian/control
>  cd ..
>  

Unfortunately, I missed to document at least one example for which I
once wrote this. I will try out the usual suspects later to see if there
are still issues (with bullseye). I agree on the direction unless it
causes massive patching needs elsewhere. In that case, making the
workaround configurable could be a compromise.

Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

RE: [PATCH 1/1] Revert "buildchroot: Do not build cross when there are no arch-specific outputs"

[Moessbauer, Felix]

Hi Jan,

> -----Original Message-----
> From: Kiszka, Jan (T RDA IOT) <jan.kiszka@siemens.com>
> Sent: Monday, January 3, 2022 8:31 AM
> To: Moessbauer, Felix (T RDA IOT SES-DE)
> <felix.moessbauer@siemens.com>; isar-users@googlegroups.com
> Subject: Re: [PATCH 1/1] Revert "buildchroot: Do not build cross when there
> are no arch-specific outputs"
> 
> On 22.12.21 13:55, Felix Moessbauer wrote:
> > This reverts commit 563986703d9a0405c70af9b50ecedede2ac59cbd.
> >
> > The reverted patch made a shortcut to simplify the handling of
> > cross-build dependencies of architecture:all packages.
> > This is not valid for the following reasons:
> >
> > 1. Just scanning the control file for 'all' is not sufficient as a
> > source package might generate both arch specific and :all packages.
> >
> > 2. This breaks architecture specific transitive build dependencies.
> > These mainly apply to correctly packaged python packages that support
> > cross building and use setuptools. By that, the transitive dependency
> > libpython3.9-minimal:<target-arch> is not attracted but instead the
> > version for the host, leading to very hard to debug build-time issues.
> >
> > In case packages do not handle their cross-build dependencies
> > correctly, we should not try to work around in ISAR, but better enforce
> upstream patches.
> >
> > Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> > ---
> >  meta/recipes-devtools/buildchroot/files/deps.sh | 6 ------
> >  1 file changed, 6 deletions(-)
> >
> > diff --git a/meta/recipes-devtools/buildchroot/files/deps.sh
> > b/meta/recipes-devtools/buildchroot/files/deps.sh
> > index 1d617bc8..ccfc460c 100644
> > --- a/meta/recipes-devtools/buildchroot/files/deps.sh
> > +++ b/meta/recipes-devtools/buildchroot/files/deps.sh
> > @@ -27,12 +27,6 @@ if [ "$3" != "--download-only" ]; then
> >          -o APT::Get::List-Cleanup="0"
> >  fi
> >
> > -# Do not set an architecture when building only 'all' (generic) packages.
> > -# This can avoid unneeded cross-build issues.
> > -if ! grep "^Architecture:" debian/control | grep -qv "all"; then
> > -    set_arch=""
> > -fi
> > -
> >  control_file=$(pwd)/debian/control
> >  cd ..
> >
> 
> Unfortunately, I missed to document at least one example for which I once
> wrote this. I will try out the usual suspects later to see if there are still issues
> (with bullseye). I agree on the direction unless it causes massive patching
> needs elsewhere. In that case, making the workaround configurable could be
> a compromise.

As the patch "[PATCH v2 0/1] meta: Reenable deps checking" is already merged, I expect that a lot of broken multi-arch packages have to be re-packaged anyways.
The deps-checking in dpkg-buildpackage is not configurable and always enabled.
By that, I doubt that the hack from above makes sense anymore as you will install packages for your host arch, but later validate them against the target arch.

The current situation (post eeefa03185e3f259d08ff94295b0981a19eddf55, without this revert) is simply broken.

I highly vote for reverting the hack.

Felix

> 
> Jan
> 
> --
> Siemens AG, T RDA IOT
> Corporate Competence Center Embedded Linux

Re: [PATCH 1/1] Revert "buildchroot: Do not build cross when there are no arch-specific outputs"

[Jan Kiszka]

On 04.01.22 11:15, Moessbauer, Felix (T CED SES-DE) wrote:
> Hi Jan,
> 
>> -----Original Message-----
>> From: Kiszka, Jan (T RDA IOT) <jan.kiszka@siemens.com>
>> Sent: Monday, January 3, 2022 8:31 AM
>> To: Moessbauer, Felix (T RDA IOT SES-DE)
>> <felix.moessbauer@siemens.com>; isar-users@googlegroups.com
>> Subject: Re: [PATCH 1/1] Revert "buildchroot: Do not build cross when there
>> are no arch-specific outputs"
>>
>> On 22.12.21 13:55, Felix Moessbauer wrote:
>>> This reverts commit 563986703d9a0405c70af9b50ecedede2ac59cbd.
>>>
>>> The reverted patch made a shortcut to simplify the handling of
>>> cross-build dependencies of architecture:all packages.
>>> This is not valid for the following reasons:
>>>
>>> 1. Just scanning the control file for 'all' is not sufficient as a
>>> source package might generate both arch specific and :all packages.
>>>
>>> 2. This breaks architecture specific transitive build dependencies.
>>> These mainly apply to correctly packaged python packages that support
>>> cross building and use setuptools. By that, the transitive dependency
>>> libpython3.9-minimal:<target-arch> is not attracted but instead the
>>> version for the host, leading to very hard to debug build-time issues.
>>>
>>> In case packages do not handle their cross-build dependencies
>>> correctly, we should not try to work around in ISAR, but better enforce
>> upstream patches.
>>>
>>> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
>>> ---
>>>  meta/recipes-devtools/buildchroot/files/deps.sh | 6 ------
>>>  1 file changed, 6 deletions(-)
>>>
>>> diff --git a/meta/recipes-devtools/buildchroot/files/deps.sh
>>> b/meta/recipes-devtools/buildchroot/files/deps.sh
>>> index 1d617bc8..ccfc460c 100644
>>> --- a/meta/recipes-devtools/buildchroot/files/deps.sh
>>> +++ b/meta/recipes-devtools/buildchroot/files/deps.sh
>>> @@ -27,12 +27,6 @@ if [ "$3" != "--download-only" ]; then
>>>          -o APT::Get::List-Cleanup="0"
>>>  fi
>>>
>>> -# Do not set an architecture when building only 'all' (generic) packages.
>>> -# This can avoid unneeded cross-build issues.
>>> -if ! grep "^Architecture:" debian/control | grep -qv "all"; then
>>> -    set_arch=""
>>> -fi
>>> -
>>>  control_file=$(pwd)/debian/control
>>>  cd ..
>>>
>>
>> Unfortunately, I missed to document at least one example for which I once
>> wrote this. I will try out the usual suspects later to see if there are still issues
>> (with bullseye). I agree on the direction unless it causes massive patching
>> needs elsewhere. In that case, making the workaround configurable could be
>> a compromise.
> 
> As the patch "[PATCH v2 0/1] meta: Reenable deps checking" is already merged, I expect that a lot of broken multi-arch packages have to be re-packaged anyways.
> The deps-checking in dpkg-buildpackage is not configurable and always enabled.
> By that, I doubt that the hack from above makes sense anymore as you will install packages for your host arch, but later validate them against the target arch.
> 
> The current situation (post eeefa03185e3f259d08ff94295b0981a19eddf55, without this revert) is simply broken.
> 
> I highly vote for reverting the hack.
> 

I don't oppose, specifically after having successfully built
jailhouse-images and meta-iot2050 with this patch applied.

Jan

-- 
Siemens AG, Technology
Competence Center Embedded Linux

RE: [PATCH 1/1] Revert "buildchroot: Do not build cross when there are no arch-specific outputs"

[Moessbauer, Felix]

Hi,

please note that this patch is a fix.
We either should have it in v0.8 or otherwise remove eeefa03185e3f259d08ff94295b0981a19eddf55 (deps checking in dpkg-buildpackage) from the release as well.
The current situation is broken (see below).

Felix

> -----Original Message-----
> From: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>
> Sent: Tuesday, January 4, 2022 1:25 PM
> To: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com>; isar-
> users@googlegroups.com
> Cc: ubely@ilbers.de
> Subject: Re: [PATCH 1/1] Revert "buildchroot: Do not build cross when there are
> no arch-specific outputs"
> 
> On 04.01.22 11:15, Moessbauer, Felix (T CED SES-DE) wrote:
> > Hi Jan,
> >
> >> -----Original Message-----
> >> From: Kiszka, Jan (T RDA IOT) <jan.kiszka@siemens.com>
> >> Sent: Monday, January 3, 2022 8:31 AM
> >> To: Moessbauer, Felix (T RDA IOT SES-DE)
> >> <felix.moessbauer@siemens.com>; isar-users@googlegroups.com
> >> Subject: Re: [PATCH 1/1] Revert "buildchroot: Do not build cross when
> >> there are no arch-specific outputs"
> >>
> >> On 22.12.21 13:55, Felix Moessbauer wrote:
> >>> This reverts commit 563986703d9a0405c70af9b50ecedede2ac59cbd.
> >>>
> >>> The reverted patch made a shortcut to simplify the handling of
> >>> cross-build dependencies of architecture:all packages.
> >>> This is not valid for the following reasons:
> >>>
> >>> 1. Just scanning the control file for 'all' is not sufficient as a
> >>> source package might generate both arch specific and :all packages.
> >>>
> >>> 2. This breaks architecture specific transitive build dependencies.
> >>> These mainly apply to correctly packaged python packages that
> >>> support cross building and use setuptools. By that, the transitive
> >>> dependency libpython3.9-minimal:<target-arch> is not attracted but
> >>> instead the version for the host, leading to very hard to debug build-time
> issues.
> >>>
> >>> In case packages do not handle their cross-build dependencies
> >>> correctly, we should not try to work around in ISAR, but better
> >>> enforce
> >> upstream patches.
> >>>
> >>> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> >>> ---
> >>>  meta/recipes-devtools/buildchroot/files/deps.sh | 6 ------
> >>>  1 file changed, 6 deletions(-)
> >>>
> >>> diff --git a/meta/recipes-devtools/buildchroot/files/deps.sh
> >>> b/meta/recipes-devtools/buildchroot/files/deps.sh
> >>> index 1d617bc8..ccfc460c 100644
> >>> --- a/meta/recipes-devtools/buildchroot/files/deps.sh
> >>> +++ b/meta/recipes-devtools/buildchroot/files/deps.sh
> >>> @@ -27,12 +27,6 @@ if [ "$3" != "--download-only" ]; then
> >>>          -o APT::Get::List-Cleanup="0"
> >>>  fi
> >>>
> >>> -# Do not set an architecture when building only 'all' (generic) packages.
> >>> -# This can avoid unneeded cross-build issues.
> >>> -if ! grep "^Architecture:" debian/control | grep -qv "all"; then
> >>> -    set_arch=""
> >>> -fi
> >>> -
> >>>  control_file=$(pwd)/debian/control
> >>>  cd ..
> >>>
> >>
> >> Unfortunately, I missed to document at least one example for which I
> >> once wrote this. I will try out the usual suspects later to see if
> >> there are still issues (with bullseye). I agree on the direction
> >> unless it causes massive patching needs elsewhere. In that case,
> >> making the workaround configurable could be a compromise.
> >
> > As the patch "[PATCH v2 0/1] meta: Reenable deps checking" is already
> merged, I expect that a lot of broken multi-arch packages have to be re-
> packaged anyways.
> > The deps-checking in dpkg-buildpackage is not configurable and always
> enabled.
> > By that, I doubt that the hack from above makes sense anymore as you will
> install packages for your host arch, but later validate them against the target
> arch.
> >
> > The current situation (post eeefa03185e3f259d08ff94295b0981a19eddf55,
> without this revert) is simply broken.
> >
> > I highly vote for reverting the hack.
> >
> 
> I don't oppose, specifically after having successfully built jailhouse-images and
> meta-iot2050 with this patch applied.
> 
> Jan
> 
> --
> Siemens AG, Technology
> Competence Center Embedded Linux

Re: [PATCH 1/1] Revert "buildchroot: Do not build cross when there are no arch-specific outputs"

[Jan Kiszka]

On 10.01.22 08:20, Moessbauer, Felix (T CED SES-DE) wrote:
> Hi,
> 
> please note that this patch is a fix.
> We either should have it in v0.8 or otherwise remove eeefa03185e3f259d08ff94295b0981a19eddf55 (deps checking in dpkg-buildpackage) from the release as well.
> The current situation is broken (see below).
> 

Ack. We are already forced to carry it locally in some layers that
depend on it.

Jan

> Felix
> 
>> -----Original Message-----
>> From: Kiszka, Jan (T CED) <jan.kiszka@siemens.com>
>> Sent: Tuesday, January 4, 2022 1:25 PM
>> To: Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com>; isar-
>> users@googlegroups.com
>> Cc: ubely@ilbers.de
>> Subject: Re: [PATCH 1/1] Revert "buildchroot: Do not build cross when there are
>> no arch-specific outputs"
>>
>> On 04.01.22 11:15, Moessbauer, Felix (T CED SES-DE) wrote:
>>> Hi Jan,
>>>
>>>> -----Original Message-----
>>>> From: Kiszka, Jan (T RDA IOT) <jan.kiszka@siemens.com>
>>>> Sent: Monday, January 3, 2022 8:31 AM
>>>> To: Moessbauer, Felix (T RDA IOT SES-DE)
>>>> <felix.moessbauer@siemens.com>; isar-users@googlegroups.com
>>>> Subject: Re: [PATCH 1/1] Revert "buildchroot: Do not build cross when
>>>> there are no arch-specific outputs"
>>>>
>>>> On 22.12.21 13:55, Felix Moessbauer wrote:
>>>>> This reverts commit 563986703d9a0405c70af9b50ecedede2ac59cbd.
>>>>>
>>>>> The reverted patch made a shortcut to simplify the handling of
>>>>> cross-build dependencies of architecture:all packages.
>>>>> This is not valid for the following reasons:
>>>>>
>>>>> 1. Just scanning the control file for 'all' is not sufficient as a
>>>>> source package might generate both arch specific and :all packages.
>>>>>
>>>>> 2. This breaks architecture specific transitive build dependencies.
>>>>> These mainly apply to correctly packaged python packages that
>>>>> support cross building and use setuptools. By that, the transitive
>>>>> dependency libpython3.9-minimal:<target-arch> is not attracted but
>>>>> instead the version for the host, leading to very hard to debug build-time
>> issues.
>>>>>
>>>>> In case packages do not handle their cross-build dependencies
>>>>> correctly, we should not try to work around in ISAR, but better
>>>>> enforce
>>>> upstream patches.
>>>>>
>>>>> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
>>>>> ---
>>>>>  meta/recipes-devtools/buildchroot/files/deps.sh | 6 ------
>>>>>  1 file changed, 6 deletions(-)
>>>>>
>>>>> diff --git a/meta/recipes-devtools/buildchroot/files/deps.sh
>>>>> b/meta/recipes-devtools/buildchroot/files/deps.sh
>>>>> index 1d617bc8..ccfc460c 100644
>>>>> --- a/meta/recipes-devtools/buildchroot/files/deps.sh
>>>>> +++ b/meta/recipes-devtools/buildchroot/files/deps.sh
>>>>> @@ -27,12 +27,6 @@ if [ "$3" != "--download-only" ]; then
>>>>>          -o APT::Get::List-Cleanup="0"
>>>>>  fi
>>>>>
>>>>> -# Do not set an architecture when building only 'all' (generic) packages.
>>>>> -# This can avoid unneeded cross-build issues.
>>>>> -if ! grep "^Architecture:" debian/control | grep -qv "all"; then
>>>>> -    set_arch=""
>>>>> -fi
>>>>> -
>>>>>  control_file=$(pwd)/debian/control
>>>>>  cd ..
>>>>>
>>>>
>>>> Unfortunately, I missed to document at least one example for which I
>>>> once wrote this. I will try out the usual suspects later to see if
>>>> there are still issues (with bullseye). I agree on the direction
>>>> unless it causes massive patching needs elsewhere. In that case,
>>>> making the workaround configurable could be a compromise.
>>>
>>> As the patch "[PATCH v2 0/1] meta: Reenable deps checking" is already
>> merged, I expect that a lot of broken multi-arch packages have to be re-
>> packaged anyways.
>>> The deps-checking in dpkg-buildpackage is not configurable and always
>> enabled.
>>> By that, I doubt that the hack from above makes sense anymore as you will
>> install packages for your host arch, but later validate them against the target
>> arch.
>>>
>>> The current situation (post eeefa03185e3f259d08ff94295b0981a19eddf55,
>> without this revert) is simply broken.
>>>
>>> I highly vote for reverting the hack.
>>>
>>
>> I don't oppose, specifically after having successfully built jailhouse-images and
>> meta-iot2050 with this patch applied.
>>
>> Jan
>>
>> --
>> Siemens AG, Technology
>> Competence Center Embedded Linux

-- 
Siemens AG, Technology
Competence Center Embedded Linux

Re: [PATCH 1/1] Revert "buildchroot: Do not build cross when there are no arch-specific outputs"

[Anton Mikanovich]

22.12.2021 15:55, Felix Moessbauer wrote:
> This reverts commit 563986703d9a0405c70af9b50ecedede2ac59cbd.
>
> The reverted patch made a shortcut to simplify the handling of cross-build
> dependencies of architecture:all packages.
> This is not valid for the following reasons:
>
> 1. Just scanning the control file for 'all' is not sufficient as
> a source package might generate both arch specific and :all packages.
>
> 2. This breaks architecture specific transitive build dependencies.
> These mainly apply to correctly packaged python packages that support
> cross building and use setuptools. By that, the transitive dependency
> libpython3.9-minimal:<target-arch> is not attracted but instead the version
> for the host, leading to very hard to debug build-time issues.
>
> In case packages do not handle their cross-build dependencies correctly, we
> should not try to work around in ISAR, but better enforce upstream patches.
>
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>

Applied to next, thanks.

-- 
Anton Mikanovich
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov