From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7192672744921628672 X-Received: by 2002:a05:6870:a414:b0:163:8701:b5a with SMTP id m20-20020a056870a41400b0016387010b5amr1162358oal.208.1675164418053; Tue, 31 Jan 2023 03:26:58 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:6808:2186:b0:374:cdbf:d1f8 with SMTP id be6-20020a056808218600b00374cdbfd1f8ls1579252oib.2.-pod-prod-gmail; Tue, 31 Jan 2023 03:26:57 -0800 (PST) X-Google-Smtp-Source: AK7set+rgbKAFjAL90NG3n4oRT79uybgfxJYuMXp7/GACBM6M8F8DaM6yvza7hbQSHZxzTKb5FST X-Received: by 2002:a05:6808:2919:b0:378:2775:39e3 with SMTP id ev25-20020a056808291900b00378277539e3mr4091426oib.8.1675164417245; Tue, 31 Jan 2023 03:26:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1675164417; cv=none; d=google.com; s=arc-20160816; b=XaHjwGYBHx0T+mB4PS+axNNF5Zf9JJPfDKw6fLqz360/Hv1up1ItwnTApIcKBKaq+1 vPAuX4ncQuFieXII6pJ/dVoq2FDNByb1qD4yc4/dUvUamh0ECtf1ffVh4O9ZbrXyaZRI QBozBKEdNI28OWEykjr81b0DVpvqI5Fqzd7UOBjlHrnQDr5/6iZTAMzNc8t9iBGp7r4W 6Gn3gNddcFjdAz60WxUCItcPrfu/g4OiP+oKWTmOs0hiI3tNaDnhK53qzSYaQMtKUXr5 xMua7YUkVV/M5Y0efwINZiDMMar+QCe8K2IrHXqXL5Ouzt2u7otMA+tGzOY5exEc07Fn ZcJA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from; bh=UEyTdmSL6ubzkXewz8sO+wnOsf7jeE3GWN7nom9oRe0=; b=AMA+3CfEK3jJUoW9pBwVWDHDD5wte5PiH3WqXCmnimaUahtlF3bsvy7UChVf4S6YPR Trna1+Mm6wBdw2uJ0A04JMXvzXL4yTv+a98CveGCsHm5ILdliUPfSAXXFIi4WZn3TKKo dt6KtWM84Gw+qz3B9J4ckWzG27z7/1xV4F2mIZXTJ7b1VQ6y1IBgnNfeNy0i5fgSn94+ HnEy1Tgfluc3DaMsak5UKXDu5s78HDqNiNfsAmBKep/+QqbJZif0n5dlTjQl7IhMRBik TQdZ5rYfRUtunEz06Jrqx9Of8gI9dcU7HOiUqjBNUy7LNsL9B8IdTkQz7NHwRUrhrymK IAuQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id p21-20020a544615000000b003783a8a36f0si586610oip.1.2023.01.31.03.26.56 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Tue, 31 Jan 2023 03:26:57 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from hp.localnet (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 30VBQrAh021038 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 31 Jan 2023 12:26:54 +0100 From: Uladzimir Bely To: isar-users@googlegroups.com Subject: Re: [PATCH v8 00/20] Migrate to Bitbake 2.0 Date: Tue, 31 Jan 2023 14:26:46 +0300 Message-ID: <21213692.4csPzL39Zc@hp> In-Reply-To: <20230125192337.86869-1-amikan@ilbers.de> References: <20230125192337.86869-1-amikan@ilbers.de> MIME-Version: 1.0 Content-Transfer-Encoding: 7Bit Content-Type: text/plain; charset="us-ascii" X-Spam-Status: No, score=-0.5 required=5.0 tests=ALL_TRUSTED,URI_NOVOWEL autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: Osh4T4NHtC0n In mail from Wednesday, 25 January 2023 22:23:17 +03 user Anton Mikanovich wrote: > This patchset moves Isar to use Bitbake 2.0 branch. > > scripts/contrib/convert-overrides.py is the script for downstream > layers overrides update. It needs some adopting for complex layers. > > Usage: > $ ./scripts/contrib/convert-overrides.py meta-customlayer > > Changes since v7: > - Rebase on next > > Changes since v6: > - Rebase on next > - Remove Debian Stretch compatibility > - Improve documentation and commit messages > - Improve changelog > - Rebuild tasks marking > > Changes since v5: > - Rebase on next > - Document sstate migration to zstd > > Changes since v4: > - Rebase on next > - Use Bitbake 2.0.5 / Yocto 4.0.5 > - Fix hash exclude variables > - Add try/catch into sstate maintenance script > - Fix overrides conversion script chmod > - Fix clean task > - Some commits reorder and rewording > > Changes since v3: > > - Rebase on next > - Add network flag for do_containerize > - Update imagetypes variables in override script > - Update RECIPE-API-CHANGELOG > > Changes since v2: > > - Fixed building of stretch targets > > Changes since v1: > > - Rebased on top of sbuild > - Resolved 'metadata is not deterministic' issues > - Update sstate class and maintenance script > - Update OE classes > > Anton Mikanovich (19): > meta: change deprecated parse calls > scripts/contrib: add override conversion script > scripts/contrib: configure override conversion script > meta-isar: set default branch names > meta: remove non recommended syntax > bitbake: update to Bitbake 2.0.5 > meta: update bitbake variables > bitbake.conf: align hash vars with openembedded > meta: mark network and sudo tasks > meta: update overrides syntax > sstate: update bbclass > bitbake.conf: declare default XZ and ZSTD options > Revert "devshell: Use different termination test to avoid warnings" > meta: align with OE-core libraries update > Revert "Revert "devshell: Use different termination test to avoid > warnings"" > CI: adapt tests to syntax change > isar-sstate: adapt sstate maintenance script > doc: require zstd tool > RECIPE-API-CHANGELOG: add tips after bitbake version update > > Roberto A. Foglietta (1): > docs: update override syntax > > RECIPE-API-CHANGELOG.md | 35 ++ > bitbake/README | 21 +- > bitbake/bin/bitbake | 4 +- > bitbake/bin/bitbake-diffsigs | 5 +- > bitbake/bin/bitbake-getvar | 50 ++ > bitbake/bin/bitbake-hashclient | 2 + > bitbake/bin/bitbake-hashserv | 2 + > bitbake/bin/bitbake-layers | 2 + > bitbake/bin/bitbake-prserv | 8 +- > bitbake/bin/bitbake-selftest | 3 + > bitbake/bin/bitbake-server | 3 +- > bitbake/bin/bitbake-worker | 23 +- > bitbake/bin/git-make-shallow | 4 + > bitbake/bin/toaster | 6 +- > bitbake/bin/toaster-eventreplay | 2 + > bitbake/conf/bitbake.conf | 6 +- > bitbake/contrib/hashserv/Dockerfile | 6 +- > bitbake/contrib/prserv/Dockerfile | 62 ++ > bitbake/contrib/vim/plugin/newbbappend.vim | 2 +- > bitbake/contrib/vim/syntax/bitbake.vim | 11 +- > bitbake/doc/Makefile | 2 +- > bitbake/doc/README | 6 +- > .../bitbake-user-manual-execution.rst | 112 ++-- > .../bitbake-user-manual-fetching.rst | 229 +++++-- > .../bitbake-user-manual-hello.rst | 60 +- > .../bitbake-user-manual-intro.rst | 64 +- > .../bitbake-user-manual-metadata.rst | 493 +++++++-------- > .../bitbake-user-manual-ref-variables.rst | 583 ++++++++++-------- > bitbake/doc/releases.rst | 84 ++- > bitbake/lib/bb/COW.py | 2 + > bitbake/lib/bb/__init__.py | 19 +- > bitbake/lib/bb/asyncrpc/__init__.py | 33 + > bitbake/lib/bb/asyncrpc/client.py | 178 ++++++ > bitbake/lib/bb/asyncrpc/serv.py | 288 +++++++++ > bitbake/lib/bb/build.py | 104 ++-- > bitbake/lib/bb/cache.py | 35 +- > bitbake/lib/bb/checksum.py | 22 +- > bitbake/lib/bb/codeparser.py | 22 +- > bitbake/lib/bb/command.py | 36 +- > bitbake/lib/bb/compress/_pipecompress.py | 196 ++++++ > bitbake/lib/bb/compress/lz4.py | 19 + > bitbake/lib/bb/compress/zstd.py | 30 + > bitbake/lib/bb/cooker.py | 298 +++++---- > bitbake/lib/bb/cookerdata.py | 47 +- > bitbake/lib/bb/daemonize.py | 44 +- > bitbake/lib/bb/data.py | 59 +- > bitbake/lib/bb/data_smart.py | 213 ++++--- > bitbake/lib/bb/event.py | 16 +- > bitbake/lib/bb/exceptions.py | 2 + > bitbake/lib/bb/fetch2/README | 57 ++ > bitbake/lib/bb/fetch2/__init__.py | 138 +++-- > bitbake/lib/bb/fetch2/crate.py | 136 ++++ > bitbake/lib/bb/fetch2/git.py | 54 +- > bitbake/lib/bb/fetch2/gitsm.py | 25 +- > bitbake/lib/bb/fetch2/npm.py | 52 +- > bitbake/lib/bb/fetch2/npmsw.py | 33 +- > bitbake/lib/bb/fetch2/osc.py | 20 +- > bitbake/lib/bb/fetch2/s3.py | 41 +- > bitbake/lib/bb/fetch2/ssh.py | 49 +- > bitbake/lib/bb/fetch2/svn.py | 10 +- > bitbake/lib/bb/fetch2/wget.py | 161 +++-- > bitbake/lib/bb/main.py | 9 +- > bitbake/lib/bb/monitordisk.py | 17 +- > bitbake/lib/bb/msg.py | 32 +- > bitbake/lib/bb/parse/__init__.py | 2 + > bitbake/lib/bb/parse/ast.py | 14 +- > bitbake/lib/bb/parse/parse_py/BBHandler.py | 7 +- > bitbake/lib/bb/parse/parse_py/ConfHandler.py | 16 +- > bitbake/lib/bb/persist_data.py | 54 +- > bitbake/lib/bb/process.py | 4 +- > bitbake/lib/bb/progress.py | 9 +- > bitbake/lib/bb/providers.py | 14 +- > bitbake/lib/bb/runqueue.py | 284 +++++---- > bitbake/lib/bb/server/process.py | 47 +- > bitbake/lib/bb/server/xmlrpcserver.py | 1 + > bitbake/lib/bb/siggen.py | 186 ++++-- > bitbake/lib/bb/taskdata.py | 14 +- > bitbake/lib/bb/tests/codeparser.py | 28 +- > bitbake/lib/bb/tests/compression.py | 100 +++ > bitbake/lib/bb/tests/cooker.py | 2 + > bitbake/lib/bb/tests/data.py | 95 +-- > .../debian/pool/main/m/minicom/index.html | 59 ++ > bitbake/lib/bb/tests/fetch.py | 533 ++++++++++------ > bitbake/lib/bb/tests/parse.py | 43 +- > .../bb/tests/runqueue-tests/conf/bitbake.conf | 2 +- > bitbake/lib/bb/tests/runqueue.py | 52 +- > bitbake/lib/bb/tests/utils.py | 20 +- > bitbake/lib/bb/tinfoil.py | 12 +- > bitbake/lib/bb/ui/buildinfohelper.py | 83 +-- > bitbake/lib/bb/ui/knotty.py | 100 +-- > bitbake/lib/bb/ui/taskexp.py | 5 + > bitbake/lib/bb/ui/uievent.py | 6 +- > bitbake/lib/bb/ui/uihelper.py | 4 +- > bitbake/lib/bb/utils.py | 161 ++++- > bitbake/lib/bblayers/__init__.py | 2 + > bitbake/lib/bblayers/action.py | 4 +- > bitbake/lib/bblayers/common.py | 2 + > bitbake/lib/bblayers/layerindex.py | 18 +- > bitbake/lib/bblayers/query.py | 10 +- > bitbake/lib/codegen.py | 6 + > bitbake/lib/hashserv/__init__.py | 66 +- > bitbake/lib/hashserv/client.py | 152 +---- > bitbake/lib/hashserv/server.py | 549 ++++++++--------- > bitbake/lib/hashserv/tests.py | 161 ++++- > bitbake/lib/layerindexlib/__init__.py | 9 +- > bitbake/lib/layerindexlib/cooker.py | 2 +- > bitbake/lib/layerindexlib/restapi.py | 4 +- > bitbake/lib/layerindexlib/tests/restapi.py | 2 +- > bitbake/lib/ply/yacc.py | 7 +- > bitbake/lib/prserv/__init__.py | 2 + > bitbake/lib/prserv/client.py | 50 ++ > bitbake/lib/prserv/db.py | 67 +- > bitbake/lib/prserv/serv.py | 542 ++++++---------- > bitbake/lib/pyinotify.py | 44 +- > .../bldcontrol/localhostbecontroller.py | 4 +- > .../management/commands/runbuilds.py | 83 ++- > .../migrations/0008_models_bigautofield.py | 48 ++ > bitbake/lib/toaster/manage.py | 2 + > .../lib/toaster/orm/fixtures/gen_fixtures.py | 445 +++++++++++++ > bitbake/lib/toaster/orm/fixtures/oe-core.xml | 48 +- > bitbake/lib/toaster/orm/fixtures/poky.xml | 118 ++-- > bitbake/lib/toaster/orm/fixtures/settings.xml | 2 +- > .../orm/management/commands/lsupdates.py | 14 +- > .../migrations/0020_models_bigautofield.py | 173 ++++++ > bitbake/lib/toaster/orm/models.py | 5 +- > .../toaster/toastergui/templates/base.html | 2 +- > .../toastergui/templates/configvars.html | 2 +- > .../toaster/toastergui/templates/landing.html | 6 +- > .../templates/landing_not_managed.html | 34 - > .../toastergui/templates/layerdetails.html | 2 +- > .../templates/package_detail_base.html | 2 +- > .../toaster/toastergui/templates/project.html | 2 +- > .../templates/project_specific.html | 2 +- > .../toastergui/templates/projectconf.html | 34 +- > bitbake/lib/toaster/toastergui/views.py | 22 +- > .../management/commands/buildimport.py | 2 +- > bitbake/lib/toaster/toastermain/settings.py | 3 + > bitbake/toaster-requirements.txt | 2 +- > doc/offline.md | 4 +- > doc/user_manual.md | 17 +- > meta-isar/classes/rpi-sdimg.bbclass | 4 +- > meta-isar/classes/ubi-ubifs-img.bbclass | 6 +- > meta-isar/conf/distro/raspios-bullseye.conf | 8 +- > meta-isar/conf/distro/ubuntu-focal.conf | 4 +- > meta-isar/conf/local.conf.sample | 8 +- > meta-isar/conf/machine/de0-nano-soc.conf | 4 +- > meta-isar/conf/machine/hikey.conf | 2 +- > meta-isar/conf/machine/phyboard-mira.conf | 2 +- > meta-isar/conf/machine/qemuamd64.conf | 14 +- > meta-isar/conf/machine/qemuarm64.conf | 2 +- > meta-isar/conf/machine/qemuriscv64.conf | 2 +- > meta-isar/conf/machine/stm32mp15x.conf | 4 +- > .../conf/multiconfig/qemuamd64-bullseye.conf | 2 +- > .../conf/multiconfig/qemuarm-bookworm.conf | 2 +- > meta-isar/recipes-app/cowsay/cowsay_git.bb | 2 +- > .../recipes-app/hello-isar/hello-isar.bb | 4 +- > meta-isar/recipes-app/libhello/libhello.bb | 4 +- > .../bootconfig-rpi/bootconfig-rpi.bb | 6 +- > .../optee-os/optee-os-stm32mp15x_3.11.0.bb | 2 +- > .../linux/linux-cip_4.4.166-cip29.bb | 2 +- > .../linux/linux-mainline_5.4.203.bb | 4 +- > .../linux/linux-phy_5.10.76-phy4.bb | 2 +- > meta/classes/base.bbclass | 1 + > meta/classes/dpkg-base.bbclass | 9 +- > meta/classes/dpkg-gbp.bbclass | 2 +- > meta/classes/dpkg-prebuilt.bbclass | 2 +- > meta/classes/dpkg.bbclass | 2 +- > meta/classes/image-locales-extension.bbclass | 2 + > meta/classes/image-tools-extension.bbclass | 1 + > meta/classes/image.bbclass | 40 +- > meta/classes/imagetypes.bbclass | 59 +- > meta/classes/imagetypes_container.bbclass | 5 +- > meta/classes/imagetypes_vm.bbclass | 10 +- > meta/classes/imagetypes_wic.bbclass | 5 +- > meta/classes/rootfs.bbclass | 7 +- > meta/classes/sdk.bbclass | 30 +- > meta/classes/sstate.bbclass | 249 ++++---- > meta/conf/bitbake.conf | 75 ++- > meta/conf/distro/debian-common.conf | 20 +- > meta/lib/oe/gpg_sign.py | 35 +- > meta/lib/oe/maketype.py | 7 +- > meta/lib/oe/patch.py | 52 +- > meta/lib/oe/path.py | 79 +++ > meta/lib/oe/sstatesig.py | 131 ++-- > meta/lib/oe/terminal.py | 33 +- > meta/lib/oe/utils.py | 67 +- > meta/recipes-bsp/barebox/barebox.inc | 2 +- > .../libubootenv/libubootenv_0.3-3.bb | 2 +- > meta/recipes-bsp/optee-os/optee-os-custom.inc | 2 +- > .../trusted-firmware-a-custom.inc | 2 +- > meta/recipes-bsp/u-boot/u-boot-custom.inc | 2 +- > .../isar-bootstrap/isar-bootstrap.inc | 14 +- > .../buildchroot/buildchroot.inc | 4 +- > .../sbuild-chroot/sbuild-chroot-host.bb | 2 +- > .../sbuild-chroot/sbuild-chroot.inc | 4 +- > meta/recipes-kernel/kselftest/kselftest.inc | 6 +- > meta/recipes-kernel/linux-module/module.inc | 2 +- > meta/recipes-kernel/linux/linux-custom.inc | 10 +- > scripts/contrib/convert-overrides.py | 145 +++++ > scripts/isar-buildenv-internal | 4 +- > scripts/isar-sstate | 70 +-- > testsuite/cibuilder.py | 6 +- > testsuite/citest.py | 2 +- > 203 files changed, 7028 insertions(+), 3385 deletions(-) > create mode 100755 bitbake/bin/bitbake-getvar > create mode 100644 bitbake/contrib/prserv/Dockerfile > create mode 100644 bitbake/lib/bb/asyncrpc/__init__.py > create mode 100644 bitbake/lib/bb/asyncrpc/client.py > create mode 100644 bitbake/lib/bb/asyncrpc/serv.py > create mode 100644 bitbake/lib/bb/compress/_pipecompress.py > create mode 100644 bitbake/lib/bb/compress/lz4.py > create mode 100644 bitbake/lib/bb/compress/zstd.py > create mode 100644 bitbake/lib/bb/fetch2/README > create mode 100644 bitbake/lib/bb/fetch2/crate.py > create mode 100644 bitbake/lib/bb/tests/compression.py > create mode 100644 > bitbake/lib/bb/tests/fetch-testdata/debian/pool/main/m/minicom/index.html > create mode 100644 bitbake/lib/prserv/client.py > create mode 100644 > bitbake/lib/toaster/bldcontrol/migrations/0008_models_bigautofield.py > create mode 100755 bitbake/lib/toaster/orm/fixtures/gen_fixtures.py create > mode 100644 bitbake/lib/toaster/orm/migrations/0020_models_bigautofield.py > delete mode 100644 > bitbake/lib/toaster/toastergui/templates/landing_not_managed.html create > mode 100755 scripts/contrib/convert-overrides.py We are going to merge this today, if there are no any urgent objections (e.g. some downsterams may not be ready).