From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6661145085876371456 X-Received: by 2002:aa7:c6c9:: with SMTP id b9mr2276208eds.7.1551689524189; Mon, 04 Mar 2019 00:52:04 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a50:9e41:: with SMTP id z59ls4446763ede.8.gmail; Mon, 04 Mar 2019 00:52:03 -0800 (PST) X-Google-Smtp-Source: APXvYqyNRoD+0KaUa0EPCslDMwRRgOD77TA5QMZvSKS7VOq4RW3SaSvWgIj7jUeo2Kd5fhjVUBMA X-Received: by 2002:aa7:c151:: with SMTP id r17mr2271659edp.9.1551689523720; Mon, 04 Mar 2019 00:52:03 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1551689523; cv=none; d=google.com; s=arc-20160816; b=zNfDsFmUgsZumy3GvgaLs0IBtrgvFAysqtagP24CYKvVdu8NO6l3zBZsBFJgHOjHZv tJ528ek/eyekUBuFs0LYLRUooM6rKHcUyuAWwaTiyovrRans11jxCyPKD4yYblQ/ay0d tkW2WWhU6wu7Rqe1bpYW5YG8gxXolCK/KV/g5Anw5lCGsMexpuLqBsUO8OY20Xl7Wjm4 b3D8TO8FsPoVJ/7FeofSI9eoAhLpqWhenVD3BW3XhFK2XccvSvleodCI+mZyVrjXY8tW STk8x8u5lFgXFTZqEfhF+kBo1pH7M6n9WnOxYdQDy+iKOAWIkezGqJNgpNih9WaIk9DW yU4g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject; bh=jERoC8MTNRYgj/ncRnj26PQDAkppFWHnDmXZdU0B9uc=; b=F65YgQnC/3hZwHjajLmAz4FlLIztcF1IVGdOsPJIf2gQebncT0hBhKQXO/sAeVXVYI 60UfBCxNWxhHhzbL54dzsr9WUuqC6JpAWduIIbH6+fr+tNPSwEvImIRy2fg2slVm53nw 6zOZmCORZfRWgOoKF9a/bntuzvqlcJfw7vtz49a9RHVNAL0eqP/0Jp7Ez1ufVu9g52qt IzS/NtZ92vfGBUaRTRLcR+7YaRsqXntQCc4A4qckqwKdZikcdWDehlPCbHK86uwh3W39 KxQcJXPeILg/5Dlvlen0pW+CSjPf/g2qKOQgkudUqqmRB6jT419YpT3y0cqw5FT0vUED PcvA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Return-Path: Received: from goliath.siemens.de (goliath.siemens.de. [192.35.17.28]) by gmr-mx.google.com with ESMTPS id r37si338324edd.2.2019.03.04.00.52.03 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 04 Mar 2019 00:52:03 -0800 (PST) Received-SPF: pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.28 as permitted sender) client-ip=192.35.17.28; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id x248q2dU022860 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 4 Mar 2019 09:52:03 +0100 Received: from [139.25.69.232] (linux-ses-ext02.ppmd.siemens.net [139.25.69.232]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id x248q11K025570; Mon, 4 Mar 2019 09:52:01 +0100 Subject: Re: [PATCH] Merge patch and unpack tasks To: Jan Kiszka , isar-users Cc: Henning Schild References: From: Claudius Heine Message-ID: <9da9415f-28f4-d67d-e1d4-910215591362@siemens.com> Date: Mon, 4 Mar 2019 09:51:58 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.4.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: D6N+zkTbdgNY Hi Jan, On 23/02/2019 11.44, Jan Kiszka wrote: > From: Jan Kiszka > > Our patch implementations does not support rolling back previously > applied patches, thus fails on patch updates without a rerun of unpack. > Avoid this by moving both steps into the same task. IMO merging multiple conceptual different tasks into one is a step back, especially with such a fundamental task as do_patch. If our implementation lacks behind the one in OE, why not copy stuff from there instead? If that is just impossible for whatever reason, then I would rather bind those tasks together with [postfuncs]. That would allow each function to be customized from other layers if necessary. regards, Claudius > > Signed-off-by: Jan Kiszka > --- > meta/classes/base.bbclass | 47 ++++++++++++++++++---- > meta/classes/dpkg-base.bbclass | 9 ++--- > meta/classes/dpkg-raw.bbclass | 2 +- > meta/classes/fit-img.bbclass | 2 +- > meta/classes/image.bbclass | 2 +- > meta/classes/patch.bbclass | 33 --------------- > meta/classes/template.bbclass | 2 +- > meta/classes/ubi-img.bbclass | 2 +- > .../isar-bootstrap/isar-bootstrap-host.bb | 2 +- > .../isar-bootstrap/isar-bootstrap-target.bb | 2 +- > .../recipes-core/isar-bootstrap/isar-bootstrap.inc | 2 +- > meta/recipes-devtools/isar-apt/isar-apt.bb | 2 +- > .../isar-cfg-localepurge/isar-cfg-localepurge.bb | 2 +- > 13 files changed, 52 insertions(+), 57 deletions(-) > delete mode 100644 meta/classes/patch.bbclass > > diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass > index 4279a68..7bdaf37 100644 > --- a/meta/classes/base.bbclass > +++ b/meta/classes/base.bbclass > @@ -1,4 +1,5 @@ > # Copyright (C) 2003 Chris Larson > +# Copyright (c) Siemens AG, 2018 > # > # Permission is hereby granted, free of charge, to any person obtaining a > # copy of this software and associated documentation files (the "Software"), > @@ -17,6 +18,8 @@ > # OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, > # ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR > # OTHER DEALINGS IN THE SOFTWARE. > +# > +# SPDX-License-Identifier: MIT > > THISDIR = "${@os.path.dirname(d.getVar('FILE', True))}" > > @@ -135,25 +138,53 @@ python do_fetch() { > > addtask fetch before do_build > > -do_unpack[dirs] = "${WORKDIR}" > -do_unpack[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > +do_unpack_and_patch[dirs] = "${WORKDIR}" > +do_unpack_and_patch[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > > -# Unpack package and put it into working directory > -python do_unpack() { > - src_uri = (d.getVar('SRC_URI', True) or "").split() > - if len(src_uri) == 0: > +# Unpack package, put it into working directory, and apply potential patches > +python do_unpack_and_patch() { > + import subprocess > + > + src_uris = (d.getVar('SRC_URI', True) or "").split() > + if len(src_uris) == 0: > return > > rootdir = d.getVar('WORKDIR', True) > + src_dir = d.getVar("S", True) > > + # unpack src_uris > try: > - fetcher = bb.fetch2.Fetch(src_uri, d) > + fetcher = bb.fetch2.Fetch(src_uris, d) > fetcher.unpack(rootdir) > except bb.fetch2.BBFetchException as e: > raise bb.build.FuncFailed(e) > + > + # apply patches > + for src_uri in src_uris: > + try: > + fetcher = bb.fetch2.Fetch([src_uri], d) > + > + apply = fetcher.ud[src_uri].parm.get("apply") > + if apply == "no": > + continue > + > + basename = fetcher.ud[src_uri].basename or "" > + if not (basename.endswith(".patch") or apply == "yes"): > + continue > + > + striplevel = fetcher.ud[src_uri].parm.get("striplevel") or "1" > + > + cmd = "patch --no-backup-if-mismatch -p " + striplevel + \ > + " --directory " + src_dir + \ > + " --input " + rootdir + '/' + basename > + bb.note(cmd) > + if subprocess.call(cmd, shell=True) != 0: > + bb.fatal("patching failed") > + except bb.fetch2.BBFetchException as e: > + raise bb.build.FuncFailed(e) > } > > -addtask unpack after do_fetch before do_build > +addtask unpack_and_patch after do_fetch before do_build > > addtask build > do_build[dirs] = "${TOPDIR}" > diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass > index 742b8ad..8ef1f06 100644 > --- a/meta/classes/dpkg-base.bbclass > +++ b/meta/classes/dpkg-base.bbclass > @@ -15,12 +15,9 @@ do_adjust_git() { > fi > } > > -addtask adjust_git after do_unpack before do_patch > +addtask adjust_git after do_unpack_and_patch before do_prepare_build > do_adjust_git[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > > -inherit patch > -addtask patch after do_adjust_git before do_build > - > SRC_APT ?= "" > > do_apt_fetch[depends] = "buildchroot-target:do_build" > @@ -40,7 +37,7 @@ do_apt_fetch() { > dpkg_undo_mounts > } > > -addtask apt_fetch after do_unpack before do_patch > +addtask apt_fetch before do_unpack_and_patch > do_apt_fetch[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" > do_apt_fetch[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > > @@ -64,7 +61,7 @@ do_prepare_build() { > true > } > > -addtask prepare_build after do_patch do_transform_template before do_build > +addtask prepare_build after do_unpack_and_patch do_transform_template before do_build > do_prepare_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > # If Isar recipes depend on each other, they typically need the package > # deployed to isar-apt > diff --git a/meta/classes/dpkg-raw.bbclass b/meta/classes/dpkg-raw.bbclass > index ea03ea4..e63ba1d 100644 > --- a/meta/classes/dpkg-raw.bbclass > +++ b/meta/classes/dpkg-raw.bbclass > @@ -17,7 +17,7 @@ do_install() { > > do_install[cleandirs] = "${D}" > do_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > -addtask install after do_unpack before do_prepare_build > +addtask install after do_unpack_and_patch before do_prepare_build > > do_prepare_build[cleandirs] += "${D}/debian" > do_prepare_build() { > diff --git a/meta/classes/fit-img.bbclass b/meta/classes/fit-img.bbclass > index edca09f..97f1d1a 100644 > --- a/meta/classes/fit-img.bbclass > +++ b/meta/classes/fit-img.bbclass > @@ -29,4 +29,4 @@ do_fit_image() { > sudo chroot ${BUILDCHROOT_DIR} /usr/bin/mkimage ${MKIMAGE_ARGS} \ > -f '${PP_WORK}/${FIT_IMAGE_SOURCE}' '${PP_DEPLOY}/${FIT_IMAGE_FILE}' > } > -addtask fit_image before do_build after do_copy_boot_files do_install_imager_deps do_unpack do_transform_template > +addtask fit_image before do_build after do_copy_boot_files do_install_imager_deps do_unpack_and_patch do_transform_template > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass > index 574fb46..a5952eb 100644 > --- a/meta/classes/image.bbclass > +++ b/meta/classes/image.bbclass > @@ -92,7 +92,7 @@ do_rootfs() { > die "No root filesystem function defined, please implement in your recipe" > } > > -addtask rootfs before do_build after do_unpack > +addtask rootfs before do_build after do_unpack_and_patch > do_rootfs[deptask] = "do_deploy_deb" > > do_mark_rootfs() { > diff --git a/meta/classes/patch.bbclass b/meta/classes/patch.bbclass > deleted file mode 100644 > index 0bc449f..0000000 > --- a/meta/classes/patch.bbclass > +++ /dev/null > @@ -1,33 +0,0 @@ > -# This software is a part of ISAR. > -# Copyright (c) Siemens AG, 2018 > - > -python do_patch() { > - import subprocess > - > - workdir = d.getVar("WORKDIR", True) + "/" > - src_dir = d.getVar("S", True) > - > - for src_uri in (d.getVar("SRC_URI", True) or "").split(): > - try: > - fetcher = bb.fetch2.Fetch([src_uri], d) > - > - apply = fetcher.ud[src_uri].parm.get("apply") > - if apply == "no": > - continue > - > - basename = fetcher.ud[src_uri].basename or "" > - if not (basename.endswith(".patch") or apply == "yes"): > - continue > - > - striplevel = fetcher.ud[src_uri].parm.get("striplevel") or "1" > - > - cmd = "patch --no-backup-if-mismatch -p " + striplevel + \ > - " --directory " + src_dir + " --input " + workdir + basename > - bb.note(cmd) > - if subprocess.call(cmd, shell=True) != 0: > - bb.fatal("patching failed") > - except bb.fetch2.BBFetchException as e: > - raise bb.build.FuncFailed(e) > -} > - > -do_patch[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > diff --git a/meta/classes/template.bbclass b/meta/classes/template.bbclass > index 324511a..8846ab0 100644 > --- a/meta/classes/template.bbclass > +++ b/meta/classes/template.bbclass > @@ -59,4 +59,4 @@ python do_transform_template() { > if process.wait() != 0: > bb.fatal("processing of template failed") > } > -addtask do_transform_template after do_unpack > +addtask do_transform_template after do_unpack_and_patch > diff --git a/meta/classes/ubi-img.bbclass b/meta/classes/ubi-img.bbclass > index f61a940..f06ab04 100644 > --- a/meta/classes/ubi-img.bbclass > +++ b/meta/classes/ubi-img.bbclass > @@ -32,4 +32,4 @@ do_ubi_image() { > sudo chroot ${BUILDCHROOT_DIR} /usr/sbin/ubinize ${UBINIZE_ARGS} \ > -o '${PP_DEPLOY}/${UBI_IMAGE_FILE}' '${PP_WORK}/${UBINIZE_CFG}' > } > -addtask ubi_image before do_build after do_copy_boot_files do_install_imager_deps do_unpack do_transform_template > +addtask ubi_image before do_build after do_copy_boot_files do_install_imager_deps do_unpack_and_patch do_transform_template > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb > index a793585..da6d5e1 100644 > --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb > @@ -40,7 +40,7 @@ python do_apt_config_prepare() { > aggregate_files(d, apt_sources_list, apt_sources_init_out) > aggregate_aptsources_list(d, apt_sources_list, apt_sources_out) > } > -addtask apt_config_prepare before do_bootstrap after do_unpack > +addtask apt_config_prepare before do_bootstrap after do_unpack_and_patch > > OVERRIDES_append = ":${@get_distro_needs_https_support(d, True)}" > > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb > index bec6fa8..b338adf 100644 > --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb > @@ -39,7 +39,7 @@ python do_apt_config_prepare() { > aggregate_files(d, apt_sources_list, apt_sources_init_out) > aggregate_aptsources_list(d, apt_sources_list, apt_sources_out) > } > -addtask apt_config_prepare before do_bootstrap after do_unpack > +addtask apt_config_prepare before do_bootstrap after do_unpack_and_patch > > OVERRIDES_append = ":${@get_distro_needs_https_support(d, False)}" > > diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > index 234d339..15995d4 100644 > --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > @@ -184,7 +184,7 @@ do_generate_keyring() { > done > fi > } > -addtask generate_keyring before do_build after do_unpack > +addtask generate_keyring before do_build after do_unpack_and_patch > > > > diff --git a/meta/recipes-devtools/isar-apt/isar-apt.bb b/meta/recipes-devtools/isar-apt/isar-apt.bb > index a959691..aea7ff7 100644 > --- a/meta/recipes-devtools/isar-apt/isar-apt.bb > +++ b/meta/recipes-devtools/isar-apt/isar-apt.bb > @@ -26,4 +26,4 @@ do_cache_config() { > fi > } > > -addtask cache_config after do_unpack before do_build > +addtask cache_config after do_unpack_and_patch before do_build > diff --git a/meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb b/meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb > index 62b4b2d..b231e1b 100644 > --- a/meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb > +++ b/meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb > @@ -55,7 +55,7 @@ do_gen_config() { > ${@get_nopurge(d)} > __EOF__ > } > -addtask gen_config after do_unpack before do_install > +addtask gen_config after do_unpack_and_patch before do_install > > do_install() { > install -v -d ${D}/usr/lib/${PN} > -- > 2.16.4 > -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de