[PATCH] wic-img: use python3.9 in bookworm

[PATCH] wic-img: use python3.9 in bookworm

[Henning Schild]

The jump to python3.10 will mean "from collections import Mapping" has
to switch to "from collections.abc import Mapping". Our bitbake still
has the old version and will not work with python 3.10. We might some
day need to fork parts of bitbake to make it work with any python3
version. But for now we can still fall back to python3.9 and do not need
to fork bitbake.

bookworm switched to python 3.10 about a month ago, since then one can
not build bookworm wic images any longer.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta/classes/wic-img.bbclass          | 7 ++++++-
 meta/conf/distro/debian-bookworm.conf | 2 ++
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/meta/classes/wic-img.bbclass b/meta/classes/wic-img.bbclass
index 7537a27b613e..5623945378eb 100644
--- a/meta/classes/wic-img.bbclass
+++ b/meta/classes/wic-img.bbclass
@@ -192,11 +192,16 @@ generate_wic_image() {
           mkdir -p /usr/bin/python3-native/
           if [ $(head -1 $(which bmaptool) | grep python3) ];then
             ln -fs /usr/bin/python3 /usr/bin/python3-native/python3
+            # python 3.10 is too new for our bitbake version
+            if [ "$( readlink /usr/bin/python3 )" = "python3.10" ]; then
+              ln -fs /usr/bin/python3.9 /usr/bin/python3-native/python3
+            fi
           else
             ln -fs /usr/bin/python2 /usr/bin/python3-native/python3
           fi
           export PATH="$BITBAKEDIR/bin:$PATH"
-          "$SCRIPTSDIR"/wic create "$WKS_FULL_PATH" \
+          /usr/bin/python3-native/python3 \
+            "$SCRIPTSDIR"/wic create "$WKS_FULL_PATH" \
             --vars "$STAGING_DIR/$MACHINE/imgdata/" \
             -o "/$WICTMP/${IMAGE_FULLNAME}.wic/" \
             --bmap \
diff --git a/meta/conf/distro/debian-bookworm.conf b/meta/conf/distro/debian-bookworm.conf
index 7e614bf2b5c3..c5b08e6aaa35 100644
--- a/meta/conf/distro/debian-bookworm.conf
+++ b/meta/conf/distro/debian-bookworm.conf
@@ -6,6 +6,8 @@ BASE_DISTRO_CODENAME = "bookworm"
 
 HOST_DISTRO ?= "debian-${BASE_DISTRO_CODENAME}"
 
+WIC_IMAGER_INSTALL += "python3.9-distutils"
+
 DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh"
 DISTRO_KERNELS ?= "4kc-malta 5kc-malta 686-pae 686 alpha-generic alpha-smp \
 	amd64 arm64 armmp-lpae armmp cloud-amd64 cloud-arm64 itanium \
-- 
2.35.1

Re: [PATCH] wic-img: use python3.9 in bookworm

[Henning Schild]

Am Wed, 20 Apr 2022 15:02:21 +0200
schrieb Henning Schild <henning.schild@siemens.com>:

> The jump to python3.10 will mean "from collections import Mapping" has
> to switch to "from collections.abc import Mapping". Our bitbake still
> has the old version and will not work with python 3.10. We might some
> day need to fork parts of bitbake to make it work with any python3
> version. But for now we can still fall back to python3.9 and do not
> need to fork bitbake.
> 
> bookworm switched to python 3.10 about a month ago, since then one can
> not build bookworm wic images any longer.

I wonder why CI did not find that like 30 days ago ... are the bookworm
tests allowed to fail, or what is that KFAIL thing?

If so, why? and is that still valid?

regards,
Henning

> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>  meta/classes/wic-img.bbclass          | 7 ++++++-
>  meta/conf/distro/debian-bookworm.conf | 2 ++
>  2 files changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes/wic-img.bbclass
> b/meta/classes/wic-img.bbclass index 7537a27b613e..5623945378eb 100644
> --- a/meta/classes/wic-img.bbclass
> +++ b/meta/classes/wic-img.bbclass
> @@ -192,11 +192,16 @@ generate_wic_image() {
>            mkdir -p /usr/bin/python3-native/
>            if [ $(head -1 $(which bmaptool) | grep python3) ];then
>              ln -fs /usr/bin/python3 /usr/bin/python3-native/python3
> +            # python 3.10 is too new for our bitbake version
> +            if [ "$( readlink /usr/bin/python3 )" = "python3.10" ];
> then
> +              ln -fs /usr/bin/python3.9
> /usr/bin/python3-native/python3
> +            fi
>            else
>              ln -fs /usr/bin/python2 /usr/bin/python3-native/python3
>            fi
>            export PATH="$BITBAKEDIR/bin:$PATH"
> -          "$SCRIPTSDIR"/wic create "$WKS_FULL_PATH" \
> +          /usr/bin/python3-native/python3 \
> +            "$SCRIPTSDIR"/wic create "$WKS_FULL_PATH" \
>              --vars "$STAGING_DIR/$MACHINE/imgdata/" \
>              -o "/$WICTMP/${IMAGE_FULLNAME}.wic/" \
>              --bmap \
> diff --git a/meta/conf/distro/debian-bookworm.conf
> b/meta/conf/distro/debian-bookworm.conf index
> 7e614bf2b5c3..c5b08e6aaa35 100644 ---
> a/meta/conf/distro/debian-bookworm.conf +++
> b/meta/conf/distro/debian-bookworm.conf @@ -6,6 +6,8 @@
> BASE_DISTRO_CODENAME = "bookworm" 
>  HOST_DISTRO ?= "debian-${BASE_DISTRO_CODENAME}"
>  
> +WIC_IMAGER_INSTALL += "python3.9-distutils"
> +
>  DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh"
>  DISTRO_KERNELS ?= "4kc-malta 5kc-malta 686-pae 686 alpha-generic
> alpha-smp \ amd64 arm64 armmp-lpae armmp cloud-amd64 cloud-arm64
> itanium \

Re: [PATCH] wic-img: use python3.9 in bookworm

[Baurzhan Ismagulov]

On Wed, Apr 20, 2022 at 03:04:48PM +0200, Henning Schild wrote:
> I wonder why CI did not find that like 30 days ago ... are the bookworm
> tests allowed to fail, or what is that KFAIL thing?
> 
> If so, why? and is that still valid?

KFAIL is "known failure" similar to that in GNU development (gdb, etc.).
Bookworm uses it since its initial introduction because the upstream is not
stable, so it's still valid IMO.

We could convert bookworm to fail (by dropping try-except and calling
self.perform_build_test() directly); in that case, any upstream problems would
have to be analyzed quickly, and we might have to wait for upstream fixes. For
our maintenance work, we'd have to implement some support for local KFAIL
tagging.

The issue is seen in http://ci.isar-build.org:8080/job/isar_next/1328/console
(green due to KFAIL), but not in earlier runs.

 (10/23)
 /build-auto/isar_next/1328/testsuite/citest.py:NoCrossTest.test_nocross_bookworm:
 ERROR: mc:qemui386-bookworm:isar-image-base-1.0-r0 do_wic_image: Error
 executing a python function in...


> > --- a/meta/classes/wic-img.bbclass
> > +++ b/meta/classes/wic-img.bbclass
> > @@ -192,11 +192,16 @@ generate_wic_image() {
> >            mkdir -p /usr/bin/python3-native/
> >            if [ $(head -1 $(which bmaptool) | grep python3) ];then
> >              ln -fs /usr/bin/python3 /usr/bin/python3-native/python3
> > +            # python 3.10 is too new for our bitbake version
> > +            if [ "$( readlink /usr/bin/python3 )" = "python3.10" ]; then
> > +              ln -fs /usr/bin/python3.9 /usr/bin/python3-native/python3

Wow, I'd expect they'd have update-alternatives for that, but seems they don't.


> > --- a/meta/conf/distro/debian-bookworm.conf
> > +++ b/meta/conf/distro/debian-bookworm.conf
> > @@ -6,6 +6,8 @@ BASE_DISTRO_CODENAME = "bookworm"
> >  
> >  HOST_DISTRO ?= "debian-${BASE_DISTRO_CODENAME}"
> >  
> > +WIC_IMAGER_INSTALL += "python3.9-distutils"

We've started testing this, but I wonder how this works. When I debootstrap
bookworm, it picks python3-distutils and installs python3.10.

Bitbake 1.50.4 has been picked as the last tagged version before overrides
syntax change. Please let us know whether bitbake 1.50.5 works; in that case we
wouldn't have to swim against the upstream.


With kind regards,
Baurzhan.

Re: [PATCH] wic-img: use python3.9 in bookworm

[Henning Schild]

Am Wed, 20 Apr 2022 18:21:08 +0200
schrieb Baurzhan Ismagulov <ibr@radix50.net>:

> On Wed, Apr 20, 2022 at 03:04:48PM +0200, Henning Schild wrote:
> > I wonder why CI did not find that like 30 days ago ... are the
> > bookworm tests allowed to fail, or what is that KFAIL thing?
> > 
> > If so, why? and is that still valid?  
> 
> KFAIL is "known failure" similar to that in GNU development (gdb,
> etc.). Bookworm uses it since its initial introduction because the
> upstream is not stable, so it's still valid IMO.

It is more than valid, with the same argument you could say you would
not want to support it at all. We do product dev on testing so we get
the most out of the stable-phase and can detect problems early enough
for upstream fixes.

> We could convert bookworm to fail (by dropping try-except and calling
> self.perform_build_test() directly); in that case, any upstream
> problems would have to be analyzed quickly, and we might have to wait
> for upstream fixes. For our maintenance work, we'd have to implement
> some support for local KFAIL tagging.

I guess we want to be aware of any upstream problem. That will cause
some friction but hopefully not too much. If concrete issues block isar
dev we could disable for a moment or merge a red pipeline. But running
such expensive CI and ignoring the result is kind of pointless, at
least i do not read CI logs regularly.
Also i think in the old CI we had bullseye without "allowed to fail"
for some time, way before it was stable. And i do not even remember
buster.

I am all for "bookwork has to succeed in CI"! We have several bookworm
layer CIs which found that downstream, now they all need to understand
and try to fix on their own.

> The issue is seen in
> http://ci.isar-build.org:8080/job/isar_next/1328/console (green due
> to KFAIL), but not in earlier runs.
> 
>  (10/23)
>  /build-auto/isar_next/1328/testsuite/citest.py:NoCrossTest.test_nocross_bookworm:
>  ERROR: mc:qemui386-bookworm:isar-image-base-1.0-r0 do_wic_image:
> Error executing a python function in...
> 
> 
> > > --- a/meta/classes/wic-img.bbclass
> > > +++ b/meta/classes/wic-img.bbclass
> > > @@ -192,11 +192,16 @@ generate_wic_image() {
> > >            mkdir -p /usr/bin/python3-native/
> > >            if [ $(head -1 $(which bmaptool) | grep python3) ];then
> > >              ln -fs /usr/bin/python3
> > > /usr/bin/python3-native/python3
> > > +            # python 3.10 is too new for our bitbake version
> > > +            if [ "$( readlink /usr/bin/python3 )" = "python3.10"
> > > ]; then
> > > +              ln -fs /usr/bin/python3.9
> > > /usr/bin/python3-native/python3  
> 
> Wow, I'd expect they'd have update-alternatives for that, but seems
> they don't.

Even if, it should not be used as long as the buildchroot is used for
building all recipes. Otherwise this step would switch from 3.10 to 3.9
and the next recipe would get 3.9.
 
> 
> > > --- a/meta/conf/distro/debian-bookworm.conf
> > > +++ b/meta/conf/distro/debian-bookworm.conf
> > > @@ -6,6 +6,8 @@ BASE_DISTRO_CODENAME = "bookworm"
> > >  
> > >  HOST_DISTRO ?= "debian-${BASE_DISTRO_CODENAME}"
> > >  
> > > +WIC_IMAGER_INSTALL += "python3.9-distutils"  
> 
> We've started testing this, but I wonder how this works. When I
> debootstrap bookworm, it picks python3-distutils and installs
> python3.10.

Yes, and you add this one as well ... also pulling in 3.9. You will
have both python versions installed.

> Bitbake 1.50.4 has been picked as the last tagged version before
> overrides syntax change. Please let us know whether bitbake 1.50.5
> works; in that case we wouldn't have to swim against the upstream.

It does work but also needs a wic bump, which also does work. I am
working on that already but am not sure how quick that will be. Maybe i
get it done today.

If you at ilbers want a quick workaround and this one here is
acceptable, please take it and i will revert it with the bumps. While
some people claim that their layers do not build anymore ... which i am
sure is right ... it can not be too pressing given they did not find it
30 days ago when bookworm switched to python 3.10.

It is a thing that would be nice to solve quickly, but not in a hurry.

regards,
Henning

> 
> With kind regards,
> Baurzhan.
> 

Re: [PATCH] wic-img: use python3.9 in bookworm

[Henning Schild]

Am Wed, 20 Apr 2022 15:02:21 +0200
schrieb Henning Schild <henning.schild@siemens.com>:

> The jump to python3.10 will mean "from collections import Mapping" has
> to switch to "from collections.abc import Mapping". Our bitbake still
> has the old version and will not work with python 3.10. We might some
> day need to fork parts of bitbake to make it work with any python3
> version. But for now we can still fall back to python3.9 and do not
> need to fork bitbake.

Ok that "new" syntax works since python3.3 and "collections" is
deprecated since. We need to bump bitbake or cherry pick its commits to
switch to ABC.

I would like to leave the bitbake version bump to others. Maybe we can
apply this patch temporarily as a quick workaround. And later bump that
bitbake and revert the workaround.

Henning

> bookworm switched to python 3.10 about a month ago, since then one can
> not build bookworm wic images any longer.
> 
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>  meta/classes/wic-img.bbclass          | 7 ++++++-
>  meta/conf/distro/debian-bookworm.conf | 2 ++
>  2 files changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes/wic-img.bbclass
> b/meta/classes/wic-img.bbclass index 7537a27b613e..5623945378eb 100644
> --- a/meta/classes/wic-img.bbclass
> +++ b/meta/classes/wic-img.bbclass
> @@ -192,11 +192,16 @@ generate_wic_image() {
>            mkdir -p /usr/bin/python3-native/
>            if [ $(head -1 $(which bmaptool) | grep python3) ];then
>              ln -fs /usr/bin/python3 /usr/bin/python3-native/python3
> +            # python 3.10 is too new for our bitbake version
> +            if [ "$( readlink /usr/bin/python3 )" = "python3.10" ];
> then
> +              ln -fs /usr/bin/python3.9
> /usr/bin/python3-native/python3
> +            fi
>            else
>              ln -fs /usr/bin/python2 /usr/bin/python3-native/python3
>            fi
>            export PATH="$BITBAKEDIR/bin:$PATH"
> -          "$SCRIPTSDIR"/wic create "$WKS_FULL_PATH" \
> +          /usr/bin/python3-native/python3 \
> +            "$SCRIPTSDIR"/wic create "$WKS_FULL_PATH" \
>              --vars "$STAGING_DIR/$MACHINE/imgdata/" \
>              -o "/$WICTMP/${IMAGE_FULLNAME}.wic/" \
>              --bmap \
> diff --git a/meta/conf/distro/debian-bookworm.conf
> b/meta/conf/distro/debian-bookworm.conf index
> 7e614bf2b5c3..c5b08e6aaa35 100644 ---
> a/meta/conf/distro/debian-bookworm.conf +++
> b/meta/conf/distro/debian-bookworm.conf @@ -6,6 +6,8 @@
> BASE_DISTRO_CODENAME = "bookworm" 
>  HOST_DISTRO ?= "debian-${BASE_DISTRO_CODENAME}"
>  
> +WIC_IMAGER_INSTALL += "python3.9-distutils"
> +
>  DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh"
>  DISTRO_KERNELS ?= "4kc-malta 5kc-malta 686-pae 686 alpha-generic
> alpha-smp \ amd64 arm64 armmp-lpae armmp cloud-amd64 cloud-arm64
> itanium \

Re: [PATCH] wic-img: use python3.9 in bookworm

[Bezdeka, Florian]

On Wed, 2022-04-20 at 16:09 +0200, Henning Schild wrote:
> Am Wed, 20 Apr 2022 15:02:21 +0200
> schrieb Henning Schild <henning.schild@siemens.com>:
> 
> > The jump to python3.10 will mean "from collections import Mapping" has
> > to switch to "from collections.abc import Mapping". Our bitbake still
> > has the old version and will not work with python 3.10. We might some
> > day need to fork parts of bitbake to make it work with any python3
> > version. But for now we can still fall back to python3.9 and do not
> > need to fork bitbake.
> 
> Ok that "new" syntax works since python3.3 and "collections" is
> deprecated since. We need to bump bitbake or cherry pick its commits to
> switch to ABC.
> 
> I would like to leave the bitbake version bump to others. Maybe we can
> apply this patch temporarily as a quick workaround. And later bump that
> bitbake and revert the workaround.

The commit message or cover letter of the last bitbake dump mentions
that the latest version did not work with ISAR. I guess we have to
learn why, fix it and bump bitbake again.

> 
> Henning
> 
> > bookworm switched to python 3.10 about a month ago, since then one can
> > not build bookworm wic images any longer.
> > 
> > Signed-off-by: Henning Schild <henning.schild@siemens.com>
> > ---
> >  meta/classes/wic-img.bbclass          | 7 ++++++-
> >  meta/conf/distro/debian-bookworm.conf | 2 ++
> >  2 files changed, 8 insertions(+), 1 deletion(-)
> > 
> > diff --git a/meta/classes/wic-img.bbclass
> > b/meta/classes/wic-img.bbclass index 7537a27b613e..5623945378eb 100644
> > --- a/meta/classes/wic-img.bbclass
> > +++ b/meta/classes/wic-img.bbclass
> > @@ -192,11 +192,16 @@ generate_wic_image() {
> >            mkdir -p /usr/bin/python3-native/
> >            if [ $(head -1 $(which bmaptool) | grep python3) ];then
> >              ln -fs /usr/bin/python3 /usr/bin/python3-native/python3
> > +            # python 3.10 is too new for our bitbake version
> > +            if [ "$( readlink /usr/bin/python3 )" = "python3.10" ];

I would expect that to break sooner or later again. Should be something
like >= python3.10 which is quite hard to realize I guess.

> > then
> > +              ln -fs /usr/bin/python3.9
> > /usr/bin/python3-native/python3
> > +            fi
> >            else
> >              ln -fs /usr/bin/python2 /usr/bin/python3-native/python3
> >            fi
> >            export PATH="$BITBAKEDIR/bin:$PATH"
> > -          "$SCRIPTSDIR"/wic create "$WKS_FULL_PATH" \
> > +          /usr/bin/python3-native/python3 \
> > +            "$SCRIPTSDIR"/wic create "$WKS_FULL_PATH" \
> >              --vars "$STAGING_DIR/$MACHINE/imgdata/" \
> >              -o "/$WICTMP/${IMAGE_FULLNAME}.wic/" \
> >              --bmap \
> > diff --git a/meta/conf/distro/debian-bookworm.conf
> > b/meta/conf/distro/debian-bookworm.conf index
> > 7e614bf2b5c3..c5b08e6aaa35 100644 ---
> > a/meta/conf/distro/debian-bookworm.conf +++
> > b/meta/conf/distro/debian-bookworm.conf @@ -6,6 +6,8 @@
> > BASE_DISTRO_CODENAME = "bookworm" 
> >  HOST_DISTRO ?= "debian-${BASE_DISTRO_CODENAME}"
> >  
> > +WIC_IMAGER_INSTALL += "python3.9-distutils"
> > +
> >  DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh"
> >  DISTRO_KERNELS ?= "4kc-malta 5kc-malta 686-pae 686 alpha-generic
> > alpha-smp \ amd64 arm64 armmp-lpae armmp cloud-amd64 cloud-arm64
> > itanium \
> 

RE: [PATCH] wic-img: use python3.9 in bookworm

[Moessbauer, Felix]

> -----Original Message-----
> From: Schild, Henning (T CED SES-DE) <henning.schild@siemens.com>
> Sent: Wednesday, April 20, 2022 4:10 PM
> To: isar-users <isar-users@googlegroups.com>
> Cc: Bezdeka, Florian (T CED SES-DE) <florian.bezdeka@siemens.com>;
> Moessbauer, Felix (T CED SES-DE) <felix.moessbauer@siemens.com>
> Subject: Re: [PATCH] wic-img: use python3.9 in bookworm
> 
> Am Wed, 20 Apr 2022 15:02:21 +0200
> schrieb Henning Schild <henning.schild@siemens.com>:
> 
> > The jump to python3.10 will mean "from collections import Mapping" has
> > to switch to "from collections.abc import Mapping". Our bitbake still
> > has the old version and will not work with python 3.10. We might some
> > day need to fork parts of bitbake to make it work with any python3
> > version. But for now we can still fall back to python3.9 and do not
> > need to fork bitbake.
> 
> Ok that "new" syntax works since python3.3 and "collections" is deprecated
> since. We need to bump bitbake or cherry pick its commits to switch to ABC.

Maybe we can just speedup the integration of the bitbake bump that is already on this list.
Just checked the tree and the compat.bb file is no longer there (because the OE patch e2be6def is already applied).
By that, the proposed bb version is not affected.

Anyways, this is a currently a road block for multiple layers on our side.

Felix

> 
> I would like to leave the bitbake version bump to others. Maybe we can apply
> this patch temporarily as a quick workaround. And later bump that bitbake and
> revert the workaround.
> 
> Henning
> 
> > bookworm switched to python 3.10 about a month ago, since then one can
> > not build bookworm wic images any longer.
> >
> > Signed-off-by: Henning Schild <henning.schild@siemens.com>
> > ---
> >  meta/classes/wic-img.bbclass          | 7 ++++++-
> >  meta/conf/distro/debian-bookworm.conf | 2 ++
> >  2 files changed, 8 insertions(+), 1 deletion(-)
> >
> > diff --git a/meta/classes/wic-img.bbclass
> > b/meta/classes/wic-img.bbclass index 7537a27b613e..5623945378eb 100644
> > --- a/meta/classes/wic-img.bbclass
> > +++ b/meta/classes/wic-img.bbclass
> > @@ -192,11 +192,16 @@ generate_wic_image() {
> >            mkdir -p /usr/bin/python3-native/
> >            if [ $(head -1 $(which bmaptool) | grep python3) ];then
> >              ln -fs /usr/bin/python3 /usr/bin/python3-native/python3
> > +            # python 3.10 is too new for our bitbake version
> > +            if [ "$( readlink /usr/bin/python3 )" = "python3.10" ];
> > then
> > +              ln -fs /usr/bin/python3.9
> > /usr/bin/python3-native/python3
> > +            fi
> >            else
> >              ln -fs /usr/bin/python2 /usr/bin/python3-native/python3
> >            fi
> >            export PATH="$BITBAKEDIR/bin:$PATH"
> > -          "$SCRIPTSDIR"/wic create "$WKS_FULL_PATH" \
> > +          /usr/bin/python3-native/python3 \
> > +            "$SCRIPTSDIR"/wic create "$WKS_FULL_PATH" \
> >              --vars "$STAGING_DIR/$MACHINE/imgdata/" \
> >              -o "/$WICTMP/${IMAGE_FULLNAME}.wic/" \
> >              --bmap \
> > diff --git a/meta/conf/distro/debian-bookworm.conf
> > b/meta/conf/distro/debian-bookworm.conf index
> > 7e614bf2b5c3..c5b08e6aaa35 100644 ---
> > a/meta/conf/distro/debian-bookworm.conf +++
> > b/meta/conf/distro/debian-bookworm.conf @@ -6,6 +6,8 @@
> > BASE_DISTRO_CODENAME = "bookworm"
> >  HOST_DISTRO ?= "debian-${BASE_DISTRO_CODENAME}"
> >
> > +WIC_IMAGER_INSTALL += "python3.9-distutils"
> > +
> >  DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh"
> >  DISTRO_KERNELS ?= "4kc-malta 5kc-malta 686-pae 686 alpha-generic
> > alpha-smp \ amd64 arm64 armmp-lpae armmp cloud-amd64 cloud-arm64
> > itanium \

Re: [PATCH] wic-img: use python3.9 in bookworm

[Henning Schild]

Am Wed, 20 Apr 2022 16:09:58 +0200
schrieb Henning Schild <henning.schild@siemens.com>:

> Am Wed, 20 Apr 2022 15:02:21 +0200
> schrieb Henning Schild <henning.schild@siemens.com>:
> 
> > The jump to python3.10 will mean "from collections import Mapping"
> > has to switch to "from collections.abc import Mapping". Our bitbake
> > still has the old version and will not work with python 3.10. We
> > might some day need to fork parts of bitbake to make it work with
> > any python3 version. But for now we can still fall back to
> > python3.9 and do not need to fork bitbake.  
> 
> Ok that "new" syntax works since python3.3 and "collections" is
> deprecated since. We need to bump bitbake or cherry pick its commits
> to switch to ABC.

We are at bitbake 1.50.4 and 1.50.5 seems to deal with several python
3.10 things, including the abc.

Henning

> I would like to leave the bitbake version bump to others. Maybe we can
> apply this patch temporarily as a quick workaround. And later bump
> that bitbake and revert the workaround.
> 
> Henning
> 
> > bookworm switched to python 3.10 about a month ago, since then one
> > can not build bookworm wic images any longer.
> > 
> > Signed-off-by: Henning Schild <henning.schild@siemens.com>
> > ---
> >  meta/classes/wic-img.bbclass          | 7 ++++++-
> >  meta/conf/distro/debian-bookworm.conf | 2 ++
> >  2 files changed, 8 insertions(+), 1 deletion(-)
> > 
> > diff --git a/meta/classes/wic-img.bbclass
> > b/meta/classes/wic-img.bbclass index 7537a27b613e..5623945378eb
> > 100644 --- a/meta/classes/wic-img.bbclass
> > +++ b/meta/classes/wic-img.bbclass
> > @@ -192,11 +192,16 @@ generate_wic_image() {
> >            mkdir -p /usr/bin/python3-native/
> >            if [ $(head -1 $(which bmaptool) | grep python3) ];then
> >              ln -fs /usr/bin/python3 /usr/bin/python3-native/python3
> > +            # python 3.10 is too new for our bitbake version
> > +            if [ "$( readlink /usr/bin/python3 )" = "python3.10" ];
> > then
> > +              ln -fs /usr/bin/python3.9
> > /usr/bin/python3-native/python3
> > +            fi
> >            else
> >              ln -fs /usr/bin/python2 /usr/bin/python3-native/python3
> >            fi
> >            export PATH="$BITBAKEDIR/bin:$PATH"
> > -          "$SCRIPTSDIR"/wic create "$WKS_FULL_PATH" \
> > +          /usr/bin/python3-native/python3 \
> > +            "$SCRIPTSDIR"/wic create "$WKS_FULL_PATH" \
> >              --vars "$STAGING_DIR/$MACHINE/imgdata/" \
> >              -o "/$WICTMP/${IMAGE_FULLNAME}.wic/" \
> >              --bmap \
> > diff --git a/meta/conf/distro/debian-bookworm.conf
> > b/meta/conf/distro/debian-bookworm.conf index
> > 7e614bf2b5c3..c5b08e6aaa35 100644 ---
> > a/meta/conf/distro/debian-bookworm.conf +++
> > b/meta/conf/distro/debian-bookworm.conf @@ -6,6 +6,8 @@
> > BASE_DISTRO_CODENAME = "bookworm" 
> >  HOST_DISTRO ?= "debian-${BASE_DISTRO_CODENAME}"
> >  
> > +WIC_IMAGER_INSTALL += "python3.9-distutils"
> > +
> >  DISTRO_CONFIG_SCRIPT ?= "debian-configscript.sh"
> >  DISTRO_KERNELS ?= "4kc-malta 5kc-malta 686-pae 686 alpha-generic
> > alpha-smp \ amd64 arm64 armmp-lpae armmp cloud-amd64 cloud-arm64
> > itanium \  
>