crossbuild broken with host-distro != target-distro

crossbuild broken with host-distro != target-distro

[Jan Kiszka]

Hi folks,

I ran into a remaining glitch for the crossbuild support: When the
target distro name differs from the host distro, buildchroot-host fails.

That initially looked like it would be easy to fix and even made sense
for a while:

diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb b/meta/recipes-devtools/buildchroot/buildchroot-host.bb
index daf92a2..01d426d 100644
--- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb
+++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb
@@ -30,5 +30,5 @@ BUILDCHROOT_PREINSTALL_append_arm64 += "binutils-aarch64-linux-gnu \
                                         crossbuild-essential-arm64"
 
 
-PARAMS = "--host-arch"
+PARAMS = "--host-arch --host-distro"
 do_build[depends] = "isar-apt:do_cache_config isar-bootstrap-host:do_bootstrap"


However, that only broke setup_root_file_system differently because it
now tried to mount the non-existing host isar-apt repo. Workaround hack:

diff --git a/meta/classes/isar-bootstrap-helper.bbclass b/meta/classes/isar-bootstrap-helper.bbclass
index d45cdd7..4ad552f 100644
--- a/meta/classes/isar-bootstrap-helper.bbclass
+++ b/meta/classes/isar-bootstrap-helper.bbclass
@@ -93,9 +93,9 @@ setup_root_file_system() {
         sudo tee "$ROOTFSDIR/etc/apt/preferences.d/isar" >/dev/null
 
     if [ ${COPYISARAPT} ]; then
-        sudo cp -Trpfx ${DEPLOY_DIR_APT}/${ROOTFS_DISTRO} $ROOTFSDIR/isar-apt
+        sudo cp -Trpfx ${DEPLOY_DIR_APT}/${DISTRO} $ROOTFSDIR/isar-apt
     else
-        sudo mount --bind ${DEPLOY_DIR_APT}/${ROOTFS_DISTRO} $ROOTFSDIR/isar-apt
+        sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} $ROOTFSDIR/isar-apt
     fi
     sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs $ROOTFSDIR/dev
     sudo mount -t proc none $ROOTFSDIR/proc


That will likely break sdkchroot, the only recipe that depends on
isar-apt-host, so far the only caller of setup_root_file_system that
passed --host-distro. But I'm not getting yet why the SDK has some own
isar-apt repo and no one else. Can you clarify and maybe even explain
how to fix the issue best?

Thanks,
Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: crossbuild broken with host-distro != target-distro

[Jan Kiszka]

On 2018-08-27 21:51, [ext] Jan Kiszka wrote:
> Hi folks,
> 
> I ran into a remaining glitch for the crossbuild support: When the
> target distro name differs from the host distro, buildchroot-host fails.
> 
> That initially looked like it would be easy to fix and even made sense
> for a while:
> 
> diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb b/meta/recipes-devtools/buildchroot/buildchroot-host.bb
> index daf92a2..01d426d 100644
> --- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb
> +++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb
> @@ -30,5 +30,5 @@ BUILDCHROOT_PREINSTALL_append_arm64 += "binutils-aarch64-linux-gnu \
>                                           crossbuild-essential-arm64"
>   
>   
> -PARAMS = "--host-arch"
> +PARAMS = "--host-arch --host-distro"
>   do_build[depends] = "isar-apt:do_cache_config isar-bootstrap-host:do_bootstrap"
> 
> 
> However, that only broke setup_root_file_system differently because it
> now tried to mount the non-existing host isar-apt repo. Workaround hack:
> 
> diff --git a/meta/classes/isar-bootstrap-helper.bbclass b/meta/classes/isar-bootstrap-helper.bbclass
> index d45cdd7..4ad552f 100644
> --- a/meta/classes/isar-bootstrap-helper.bbclass
> +++ b/meta/classes/isar-bootstrap-helper.bbclass
> @@ -93,9 +93,9 @@ setup_root_file_system() {
>           sudo tee "$ROOTFSDIR/etc/apt/preferences.d/isar" >/dev/null
>   
>       if [ ${COPYISARAPT} ]; then
> -        sudo cp -Trpfx ${DEPLOY_DIR_APT}/${ROOTFS_DISTRO} $ROOTFSDIR/isar-apt
> +        sudo cp -Trpfx ${DEPLOY_DIR_APT}/${DISTRO} $ROOTFSDIR/isar-apt
>       else
> -        sudo mount --bind ${DEPLOY_DIR_APT}/${ROOTFS_DISTRO} $ROOTFSDIR/isar-apt
> +        sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} $ROOTFSDIR/isar-apt
>       fi
>       sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs $ROOTFSDIR/dev
>       sudo mount -t proc none $ROOTFSDIR/proc
> 
> 
> That will likely break sdkchroot, the only recipe that depends on
> isar-apt-host, so far the only caller of setup_root_file_system that
> passed --host-distro. But I'm not getting yet why the SDK has some own
> isar-apt repo and no one else. Can you clarify and maybe even explain
> how to fix the issue best?
> 
> Thanks,
> Jan
> 

Could you comment on this bug? It's a blocker here.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: crossbuild broken with host-distro != target-distro

[Maxim Yu. Osipov]

Hi Jan,


On 08/30/2018 11:24 AM, Jan Kiszka wrote:
> On 2018-08-27 21:51, [ext] Jan Kiszka wrote:
>> Hi folks,
>>
>> I ran into a remaining glitch for the crossbuild support: When the
>> target distro name differs from the host distro, buildchroot-host fails.
>>
>> That initially looked like it would be easy to fix and even made sense
>> for a while:
>>
>> diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb 
>> b/meta/recipes-devtools/buildchroot/buildchroot-host.bb
>> index daf92a2..01d426d 100644
>> --- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb
>> +++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb
>> @@ -30,5 +30,5 @@ BUILDCHROOT_PREINSTALL_append_arm64 += 
>> "binutils-aarch64-linux-gnu \
>>                                           crossbuild-essential-arm64"
>> -PARAMS = "--host-arch"
>> +PARAMS = "--host-arch --host-distro"
>>   do_build[depends] = "isar-apt:do_cache_config 
>> isar-bootstrap-host:do_bootstrap"
>>
>>
>> However, that only broke setup_root_file_system differently because it
>> now tried to mount the non-existing host isar-apt repo. Workaround hack:
>>
>> diff --git a/meta/classes/isar-bootstrap-helper.bbclass 
>> b/meta/classes/isar-bootstrap-helper.bbclass
>> index d45cdd7..4ad552f 100644
>> --- a/meta/classes/isar-bootstrap-helper.bbclass
>> +++ b/meta/classes/isar-bootstrap-helper.bbclass
>> @@ -93,9 +93,9 @@ setup_root_file_system() {
>>           sudo tee "$ROOTFSDIR/etc/apt/preferences.d/isar" >/dev/null
>>       if [ ${COPYISARAPT} ]; then
>> -        sudo cp -Trpfx ${DEPLOY_DIR_APT}/${ROOTFS_DISTRO} 
>> $ROOTFSDIR/isar-apt
>> +        sudo cp -Trpfx ${DEPLOY_DIR_APT}/${DISTRO} $ROOTFSDIR/isar-apt
>>       else
>> -        sudo mount --bind ${DEPLOY_DIR_APT}/${ROOTFS_DISTRO} 
>> $ROOTFSDIR/isar-apt
>> +        sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} 
>> $ROOTFSDIR/isar-apt
>>       fi
>>       sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs $ROOTFSDIR/dev
>>       sudo mount -t proc none $ROOTFSDIR/proc
>>
>>
>> That will likely break sdkchroot, the only recipe that depends on
>> isar-apt-host, so far the only caller of setup_root_file_system that
>> passed --host-distro. But I'm not getting yet why the SDK has some own
>> isar-apt repo and no one else. Can you clarify and maybe even explain
>> how to fix the issue best?

The initial idea is that sdk may be distributed to other hosts -
so we need to copy local isar-apt to sdk chroot.
Actually copying of isar-apt here is redundant, as the same is done 
later in do_populate_sdk to get all latest prebuilt packages for target 
distro.


>> Thanks,
>> Jan
>>
> 
> Could you comment on this bug? It's a blocker here.

I'm OK with your patch - it's reasonable. I've throughly tested it 
together with my v2 sdk: Move deployment to populate_sdk.

Just sent a v2 series.

Kind regards,
Maxim.

-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: crossbuild broken with host-distro != target-distro

[Jan Kiszka]

On 2018-09-05 10:31, Maxim Yu. Osipov wrote:
> Hi Jan,
> 
> 
> On 08/30/2018 11:24 AM, Jan Kiszka wrote:
>> On 2018-08-27 21:51, [ext] Jan Kiszka wrote:
>>> Hi folks,
>>>
>>> I ran into a remaining glitch for the crossbuild support: When the
>>> target distro name differs from the host distro, buildchroot-host fails.
>>>
>>> That initially looked like it would be easy to fix and even made sense
>>> for a while:
>>>
>>> diff --git a/meta/recipes-devtools/buildchroot/buildchroot-host.bb 
>>> b/meta/recipes-devtools/buildchroot/buildchroot-host.bb
>>> index daf92a2..01d426d 100644
>>> --- a/meta/recipes-devtools/buildchroot/buildchroot-host.bb
>>> +++ b/meta/recipes-devtools/buildchroot/buildchroot-host.bb
>>> @@ -30,5 +30,5 @@ BUILDCHROOT_PREINSTALL_append_arm64 += 
>>> "binutils-aarch64-linux-gnu \
>>>                                           crossbuild-essential-arm64"
>>> -PARAMS = "--host-arch"
>>> +PARAMS = "--host-arch --host-distro"
>>>   do_build[depends] = "isar-apt:do_cache_config 
>>> isar-bootstrap-host:do_bootstrap"
>>>
>>>
>>> However, that only broke setup_root_file_system differently because it
>>> now tried to mount the non-existing host isar-apt repo. Workaround hack:
>>>
>>> diff --git a/meta/classes/isar-bootstrap-helper.bbclass 
>>> b/meta/classes/isar-bootstrap-helper.bbclass
>>> index d45cdd7..4ad552f 100644
>>> --- a/meta/classes/isar-bootstrap-helper.bbclass
>>> +++ b/meta/classes/isar-bootstrap-helper.bbclass
>>> @@ -93,9 +93,9 @@ setup_root_file_system() {
>>>           sudo tee "$ROOTFSDIR/etc/apt/preferences.d/isar" >/dev/null
>>>       if [ ${COPYISARAPT} ]; then
>>> -        sudo cp -Trpfx ${DEPLOY_DIR_APT}/${ROOTFS_DISTRO} 
>>> $ROOTFSDIR/isar-apt
>>> +        sudo cp -Trpfx ${DEPLOY_DIR_APT}/${DISTRO} $ROOTFSDIR/isar-apt
>>>       else
>>> -        sudo mount --bind ${DEPLOY_DIR_APT}/${ROOTFS_DISTRO} 
>>> $ROOTFSDIR/isar-apt
>>> +        sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} 
>>> $ROOTFSDIR/isar-apt
>>>       fi
>>>       sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs $ROOTFSDIR/dev
>>>       sudo mount -t proc none $ROOTFSDIR/proc
>>>
>>>
>>> That will likely break sdkchroot, the only recipe that depends on
>>> isar-apt-host, so far the only caller of setup_root_file_system that
>>> passed --host-distro. But I'm not getting yet why the SDK has some own
>>> isar-apt repo and no one else. Can you clarify and maybe even explain
>>> how to fix the issue best?
> 
> The initial idea is that sdk may be distributed to other hosts -
> so we need to copy local isar-apt to sdk chroot.
> Actually copying of isar-apt here is redundant, as the same is done 
> later in do_populate_sdk to get all latest prebuilt packages for target 
> distro.

Please explain again the different roles of isar-apt vs. isar-apt-host, 
why we need both.

Then we need some wording about why that ROOTFS_DISTRO -> DISTRO is 
correct. We can't merge such changes without explanations. No one will 
understand the code and its history this way.

Jan

> 
> 
>>> Thanks,
>>> Jan
>>>
>>
>> Could you comment on this bug? It's a blocker here.
> 
> I'm OK with your patch - it's reasonable. I've throughly tested it 
> together with my v2 sdk: Move deployment to populate_sdk.
> 
> Just sent a v2 series.
> 
> Kind regards,
> Maxim.
> 

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux