Re: [PATCH] ci-build: Protect bullseye container with KFAIL

[Jan Kiszka <jan.kiszka@siemens.com>]

On 30.04.21 09:49, Moessbauer, Felix (T RDA IOT SES-DE) wrote:
> Hi Jan, Anton,
> 
> Actually I don't think the issue is related to binfmt_misc.
> If I get it right, binfmt_misc just dispatches and starts the arm binary with qemu-arm-static.
> 
> The bug (missing handling of syscall) is in the qemu binary itself.
> Neither mmdebstrap nor namespace support in binfmt_misc can help here.

The problem falls into two parts:

 - we need a newer qemu-user-static package than what buster ships in
   order to extend/fix the syscall emulation
 - binfmt_misc needs to use the right binary, not a stale one due to
   "fix binary" [1]

For former is what kas already does (buster-backports package). But
maybe our version in 2.4 is now too old. I'll check locally.

The latter is a problem when sharing the building kernel with multiple
jobs AND permitting those jobs to bring their own qemu-user-static
binary. That is the model of kas-container, to avoid contamination from
the docker host distro (which may not even be Debian). But it tends to
breaks due to missing isolation of the containers /wrt binfmt_misc.

BTW, if there is a way to detect that a wrong version of
qemu-user-static would actually be used due to "fix binary", I'm happy
to take at least a warning patch for kas-isar (docker entrypoint),
possibly even an overwrite to upgrade (although that may break builds
running in parallel).

Jan

[1] https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html

> 
> The good thing is, that we can detect the broken qemu version and issue a warning.
> But that just spams logs as the do_install_builddeps is a heavily used task...
> 
> Best regards,
> Felix
> 
>> -----Original Message-----
>> From: Jan Kiszka <jan.kiszka@siemens.com>
>> Sent: Thursday, April 29, 2021 7:46 PM
>> To: Moessbauer, Felix (T RDA IOT SES-DE) <felix.moessbauer@siemens.com>;
>> Anton Mikanovich <amikan@ilbers.de>
>> Subject: Re: [PATCH] ci-build: Protect bullseye container with KFAIL
>>
>> On 29.04.21 15:42, Moessbauer, Felix (T RDA IOT SES-DE) wrote:
>>> Hi Anton,
>>>
>>> I can confirm that the CI patch does not solve the issue (on mc:qemuarm64-
>> focal).
>>> In case of the gitlab-ci, the KAS image provides the qemu-user-static.
>>
>> ...as along as the host didn't have an older version already used (e.g.
>> from an older container). By default, binfmt_misc is sticky /wrt the first used
>> interpreter binary. When unsure, clean all binfmt settings, then use the latest
>> container image.
>>
>>>
>>> I just tried to install qemu-user-static from buster-backports into the KAS
>> container and that solved the issue.
>>> Maybe Jan might have a look into that. IMO this would be way better than
>> trying to work-around in ISAR.
>>
>> Sure, this is a host (or build container) issue, nothing for Isar itself.
>>
>> Eventually, mmdebstrap in chrootless mode should help to overcome such
>> issues. Or proper namespace support for binfmt_misc.
>>
>> Jan
>>
>>>
>>> Best regards,
>>> Felix
>>>
>>>> -----Original Message-----
>>>> From: isar-users@googlegroups.com <isar-users@googlegroups.com> On
>>>> Behalf Of Anton Mikanovich
>>>> Sent: Friday, April 9, 2021 10:15 AM
>>>> To: isar-users@googlegroups.com
>>>> Cc: Anton Mikanovich <amikan@ilbers.de>
>>>> Subject: [PATCH] ci-build: Protect bullseye container with KFAIL
>>>>
>>>> Bullseye building issue still not resolved, so container-bullseye fails with error:
>>>>
>>>>> fakeroot debian/rules clean
>>>>> semop(1): encountered an error: Function not implemented
>>>>
>>>> This issue should be already fixed in host qemu/1:5.0-9 but we still
>>>> experience the same with qemu/1:5.2+dfsg-3~bpo10+1. Further research
>>>> will be done, so temporary fix for ci-build.sh introduced.
>>>>
>>>> Bug report:
>>>> https://bug
>>>>
>> s.deb%2F&amp;data=04%7C01%7Cfelix.moessbauer%40siemens.com%7C92bd
>> 09c5
>>>>
>> 68bb45136a1008d90b37b6b5%7C38ae3bcd95794fd4addab42e1495d55a%7C1%
>> 7C0%7
>>>>
>> C637553156176955322%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwM
>> DAiLCJQI
>>>>
>> joiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Q9A9N5
>> m%2B
>>>> BTq%2BzEJI4Sfkp6aInSDj8qgPZ3SZlh6t7WE%3D&amp;reserved=0
>>>> ian.org%2Fcgi-
>>>>
>> bin%2Fbugreport.cgi%3Fbug%3D965109&amp;data=04%7C01%7Cfelix.moessba
>>>>
>> uer%40siemens.com%7C4c4f7801dd2b4db1c5a808d8fb2fa8dd%7C38ae3bcd95
>>>>
>> 794fd4addab42e1495d55a%7C1%7C0%7C637535529408448819%7CUnknown%
>>>>
>> 7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
>>>>
>> CJXVCI6Mn0%3D%7C1000&amp;sdata=pvHkywD1yglmoyG7TOjSaTTYWtFOZL%
>>>> 2FqAlrG4UOjpQM%3D&amp;reserved=0
>>>>
>>>> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
>>>> ---
>>>>  scripts/ci_build.sh | 9 +++++++++
>>>>  1 file changed, 9 insertions(+)
>>>>
>>>> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh index
>>>> 6445b10..adc22e4
>>>> 100755
>>>> --- a/scripts/ci_build.sh
>>>> +++ b/scripts/ci_build.sh
>>>> @@ -58,6 +58,9 @@ TARGETS_SET_BULLSEYE="\
>> TARGETS_CONTAINERS="\
>>>>      mc:container-stretch:isar-image-base \
>>>>      mc:container-buster:isar-image-base \
>>>> +"
>>>> +
>>>> +TARGETS_CONTAINERS_BULLSEYE="\
>>>>      mc:container-bullseye:isar-image-base \  "
>>>>
>>>> @@ -259,6 +262,12 @@ sed -i -e 's/\(IMAGE_INSTALL = .*\)
>>>> example-module- ${KERNEL_NAME}\(.*\)/\1\2/g'
>>>>  sed -i -e 's/\(IMAGE_INSTALL = .*\) enable-fsck\(.*\)/\1\2/g'
>>>> conf/local.conf bitbake $BB_ARGS $TARGETS_CONTAINERS  while [ -e
>>>> bitbake.sock ]; do sleep 1; done
>>>> +if bitbake $BB_ARGS $TARGETS_CONTAINERS_BULLSEYE; then
>>>> +    echo "bullseye container: PASSED"
>>>> +else
>>>> +    echo "bullseye container: KFAIL"
>>>> +fi
>>>> +while [ -e bitbake.sock ]; do sleep 1; done
>>>>  # and SDK container image creation
>>>>  echo 'SDK_FORMATS = "docker-archive"' >> conf/local.conf  bitbake
>>>> $BB_ARGS -c do_populate_sdk mc:container-stretch:isar-image-base
>>>> --
>>>> 2.20.1
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "isar-users" group.
>>>> To unsubscribe from this group and stop receiving emails from it,
>>>> send an email to isar-users+unsubscribe@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://gro
>>>>
>> ups.g%2F&amp;data=04%7C01%7Cfelix.moessbauer%40siemens.com%7C92bd
>> 09c5
>>>>
>> 68bb45136a1008d90b37b6b5%7C38ae3bcd95794fd4addab42e1495d55a%7C1%
>> 7C0%7
>>>>
>> C637553156176955322%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwM
>> DAiLCJQI
>>>>
>> joiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=u3Q8XDu
>> KM0
>>>> IxNW3DetVika0M3D73yWxn8Nd3TIfpb1Q%3D&amp;reserved=0
>>>> oogle.com%2Fd%2Fmsgid%2Fisar-users%2F20210409081520.399406-1-
>>>>
>> amikan%2540ilbers.de&amp;data=04%7C01%7Cfelix.moessbauer%40siemens.c
>>>>
>> om%7C4c4f7801dd2b4db1c5a808d8fb2fa8dd%7C38ae3bcd95794fd4addab42e1
>>>>
>> 495d55a%7C1%7C0%7C637535529408448819%7CUnknown%7CTWFpbGZsb3d8
>>>>
>> eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%
>>>>
>> 7C1000&amp;sdata=th0wxFgLnp3sT51ifrusbltvJ7NKgsDnGUtVXEfXHQs%3D&am
>>>> p;reserved=0.


-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux