Re: [PATCH] ci-build: Protect bullseye container with KFAIL

[Jan Kiszka <jan.kiszka@siemens.com>]

On 30.04.21 19:57, Jan Kiszka wrote:
> On 30.04.21 09:49, Moessbauer, Felix (T RDA IOT SES-DE) wrote:
>> Hi Jan, Anton,
>>
>> Actually I don't think the issue is related to binfmt_misc.
>> If I get it right, binfmt_misc just dispatches and starts the arm binary with qemu-arm-static.
>>
>> The bug (missing handling of syscall) is in the qemu binary itself.
>> Neither mmdebstrap nor namespace support in binfmt_misc can help here.
> 
> The problem falls into two parts:
> 
>  - we need a newer qemu-user-static package than what buster ships in
>    order to extend/fix the syscall emulation
>  - binfmt_misc needs to use the right binary, not a stale one due to
>    "fix binary" [1]
> 
> For former is what kas already does (buster-backports package). But
> maybe our version in 2.4 is now too old. I'll check locally.
> 
> The latter is a problem when sharing the building kernel with multiple
> jobs AND permitting those jobs to bring their own qemu-user-static
> binary. That is the model of kas-container, to avoid contamination from
> the docker host distro (which may not even be Debian). But it tends to
> breaks due to missing isolation of the containers /wrt binfmt_misc.
> 
> BTW, if there is a way to detect that a wrong version of
> qemu-user-static would actually be used due to "fix binary", I'm happy
> to take at least a warning patch for kas-isar (docker entrypoint),
> possibly even an overwrite to upgrade (although that may break builds
> running in parallel).
> 

We have an extra mess here:

Because of an incompletely definition of machine/container.conf, we were
building containers for random archs, armhf in this case. I've a patch
for that under test, confining containers to amd64 for now (but that
could be enhanced in the future). It should move
container-amd64-bullseye out of the focus.

Still, I would expect that qemuarmhf-bullseye has the qemu-user-static
issue as well when an incorrect version is fixed-installed on the host.
Validating...

Jan

> Jan
> 
> [1] https://www.kernel.org/doc/html/latest/admin-guide/binfmt-misc.html
> 
>>
>> The good thing is, that we can detect the broken qemu version and issue a warning.
>> But that just spams logs as the do_install_builddeps is a heavily used task...
>>
>> Best regards,
>> Felix
>>
>>> -----Original Message-----
>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>> Sent: Thursday, April 29, 2021 7:46 PM
>>> To: Moessbauer, Felix (T RDA IOT SES-DE) <felix.moessbauer@siemens.com>;
>>> Anton Mikanovich <amikan@ilbers.de>
>>> Subject: Re: [PATCH] ci-build: Protect bullseye container with KFAIL
>>>
>>> On 29.04.21 15:42, Moessbauer, Felix (T RDA IOT SES-DE) wrote:
>>>> Hi Anton,
>>>>
>>>> I can confirm that the CI patch does not solve the issue (on mc:qemuarm64-
>>> focal).
>>>> In case of the gitlab-ci, the KAS image provides the qemu-user-static.
>>>
>>> ...as along as the host didn't have an older version already used (e.g.
>>> from an older container). By default, binfmt_misc is sticky /wrt the first used
>>> interpreter binary. When unsure, clean all binfmt settings, then use the latest
>>> container image.
>>>
>>>>
>>>> I just tried to install qemu-user-static from buster-backports into the KAS
>>> container and that solved the issue.
>>>> Maybe Jan might have a look into that. IMO this would be way better than
>>> trying to work-around in ISAR.
>>>
>>> Sure, this is a host (or build container) issue, nothing for Isar itself.
>>>
>>> Eventually, mmdebstrap in chrootless mode should help to overcome such
>>> issues. Or proper namespace support for binfmt_misc.
>>>
>>> Jan
>>>
>>>>
>>>> Best regards,
>>>> Felix
>>>>
>>>>> -----Original Message-----
>>>>> From: isar-users@googlegroups.com <isar-users@googlegroups.com> On
>>>>> Behalf Of Anton Mikanovich
>>>>> Sent: Friday, April 9, 2021 10:15 AM
>>>>> To: isar-users@googlegroups.com
>>>>> Cc: Anton Mikanovich <amikan@ilbers.de>
>>>>> Subject: [PATCH] ci-build: Protect bullseye container with KFAIL
>>>>>
>>>>> Bullseye building issue still not resolved, so container-bullseye fails with error:
>>>>>
>>>>>> fakeroot debian/rules clean
>>>>>> semop(1): encountered an error: Function not implemented
>>>>>
>>>>> This issue should be already fixed in host qemu/1:5.0-9 but we still
>>>>> experience the same with qemu/1:5.2+dfsg-3~bpo10+1. Further research
>>>>> will be done, so temporary fix for ci-build.sh introduced.
>>>>>
>>>>> Bug report:
>>>>> https://bug
>>>>>
>>> s.deb%2F&amp;data=04%7C01%7Cfelix.moessbauer%40siemens.com%7C92bd
>>> 09c5
>>>>>
>>> 68bb45136a1008d90b37b6b5%7C38ae3bcd95794fd4addab42e1495d55a%7C1%
>>> 7C0%7
>>>>>
>>> C637553156176955322%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwM
>>> DAiLCJQI
>>>>>
>>> joiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=Q9A9N5
>>> m%2B
>>>>> BTq%2BzEJI4Sfkp6aInSDj8qgPZ3SZlh6t7WE%3D&amp;reserved=0
>>>>> ian.org%2Fcgi-
>>>>>
>>> bin%2Fbugreport.cgi%3Fbug%3D965109&amp;data=04%7C01%7Cfelix.moessba
>>>>>
>>> uer%40siemens.com%7C4c4f7801dd2b4db1c5a808d8fb2fa8dd%7C38ae3bcd95
>>>>>
>>> 794fd4addab42e1495d55a%7C1%7C0%7C637535529408448819%7CUnknown%
>>>>>
>>> 7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiL
>>>>>
>>> CJXVCI6Mn0%3D%7C1000&amp;sdata=pvHkywD1yglmoyG7TOjSaTTYWtFOZL%
>>>>> 2FqAlrG4UOjpQM%3D&amp;reserved=0
>>>>>
>>>>> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
>>>>> ---
>>>>>  scripts/ci_build.sh | 9 +++++++++
>>>>>  1 file changed, 9 insertions(+)
>>>>>
>>>>> diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh index
>>>>> 6445b10..adc22e4
>>>>> 100755
>>>>> --- a/scripts/ci_build.sh
>>>>> +++ b/scripts/ci_build.sh
>>>>> @@ -58,6 +58,9 @@ TARGETS_SET_BULLSEYE="\
>>> TARGETS_CONTAINERS="\
>>>>>      mc:container-stretch:isar-image-base \
>>>>>      mc:container-buster:isar-image-base \
>>>>> +"
>>>>> +
>>>>> +TARGETS_CONTAINERS_BULLSEYE="\
>>>>>      mc:container-bullseye:isar-image-base \  "
>>>>>
>>>>> @@ -259,6 +262,12 @@ sed -i -e 's/\(IMAGE_INSTALL = .*\)
>>>>> example-module- ${KERNEL_NAME}\(.*\)/\1\2/g'
>>>>>  sed -i -e 's/\(IMAGE_INSTALL = .*\) enable-fsck\(.*\)/\1\2/g'
>>>>> conf/local.conf bitbake $BB_ARGS $TARGETS_CONTAINERS  while [ -e
>>>>> bitbake.sock ]; do sleep 1; done
>>>>> +if bitbake $BB_ARGS $TARGETS_CONTAINERS_BULLSEYE; then
>>>>> +    echo "bullseye container: PASSED"
>>>>> +else
>>>>> +    echo "bullseye container: KFAIL"
>>>>> +fi
>>>>> +while [ -e bitbake.sock ]; do sleep 1; done
>>>>>  # and SDK container image creation
>>>>>  echo 'SDK_FORMATS = "docker-archive"' >> conf/local.conf  bitbake
>>>>> $BB_ARGS -c do_populate_sdk mc:container-stretch:isar-image-base
>>>>> --
>>>>> 2.20.1
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "isar-users" group.
>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>> send an email to isar-users+unsubscribe@googlegroups.com.
>>>>> To view this discussion on the web visit
>>>>> https://gro
>>>>>
>>> ups.g%2F&amp;data=04%7C01%7Cfelix.moessbauer%40siemens.com%7C92bd
>>> 09c5
>>>>>
>>> 68bb45136a1008d90b37b6b5%7C38ae3bcd95794fd4addab42e1495d55a%7C1%
>>> 7C0%7
>>>>>
>>> C637553156176955322%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwM
>>> DAiLCJQI
>>>>>
>>> joiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=u3Q8XDu
>>> KM0
>>>>> IxNW3DetVika0M3D73yWxn8Nd3TIfpb1Q%3D&amp;reserved=0
>>>>> oogle.com%2Fd%2Fmsgid%2Fisar-users%2F20210409081520.399406-1-
>>>>>
>>> amikan%2540ilbers.de&amp;data=04%7C01%7Cfelix.moessbauer%40siemens.c
>>>>>
>>> om%7C4c4f7801dd2b4db1c5a808d8fb2fa8dd%7C38ae3bcd95794fd4addab42e1
>>>>>
>>> 495d55a%7C1%7C0%7C637535529408448819%7CUnknown%7CTWFpbGZsb3d8
>>>>>
>>> eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%
>>>>>
>>> 7C1000&amp;sdata=th0wxFgLnp3sT51ifrusbltvJ7NKgsDnGUtVXEfXHQs%3D&am
>>>>> p;reserved=0.
> 
> 

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux