From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6661145085876371456 X-Received: by 2002:a2e:4784:: with SMTP id u126mr1112091lja.22.1552062865926; Fri, 08 Mar 2019 08:34:25 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:5c06:: with SMTP id q6ls1225286ljb.9.gmail; Fri, 08 Mar 2019 08:34:25 -0800 (PST) X-Google-Smtp-Source: APXvYqzQJPC40UTOAqkkKA6rAIWSVCQW7KPJ557i2MblyymWNgaLdBqO4akwkC+vyGQ33uQf4Ti9 X-Received: by 2002:a2e:85cd:: with SMTP id h13mr1042872ljj.8.1552062865179; Fri, 08 Mar 2019 08:34:25 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1552062865; cv=none; d=google.com; s=arc-20160816; b=Xz86EaGdkqvX4tD/g2reFQT8sF6J87RQF7Dr8EAC9OLwJqHl03K+6liCrOGIrJWu7U xhYeCqnvXdlcORB4KMLkEvEBVIdDGevZ9cpGtjq0N7jn61tNM+44Gz3T/sWhbC7+lvOc JMES3nD3ASU+dyCogtsVPVzwfQTzmadCBcm9osTnSghrseI0JtjXckoTvdHj7JQfhfY8 8c0vT0FBBKxG3CLWSgNsz/zLpwEIaKGkV+2EPqLFsaGncIZpEcTFE54pV44YiZf4LKIN wAK62BMCpaAHUMsf38RQm4Z/dgK0hl4RgDE2XuvLQmtBeUA8gMecgANuDFrmTRElu3fg /3ag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject; bh=fkD5D8naKAywyZVR0yfw+UBDtPp22KDJJ2T+NJktf6k=; b=iZr87Ljj6m7+3Od1V0j3kTQsUtdzicqOHQzrk9OSwGEntcB1/Cq27XRQDkho21+fBk ihuU0p2U+DrFxJugvtCyObLQtGZeu+/vlUhb5IWBZOT77nKTWWWnK16s+095cqkiwi4O o4C9adlVqOipWfNV6OyU4fIXdL/7yqHxTSouf4rfmtucggHfnT2oAYFtKG4OnHlvaXVG 8aUW3zdNIW81ZcOiQ3p2mzDlwe6y558dLlLbdEHLhHyzI3iPQl0ySNbq+dS9jwBOeCQ1 COsjZGWq+t129Bb8/5hp1JyXozW7mVn6FuCmTncsFI/++nul1iqar+LfTnwQXUQALF9A wWnw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Return-Path: Received: from gecko.sbs.de (gecko.sbs.de. [194.138.37.40]) by gmr-mx.google.com with ESMTPS id n21si389242lfl.2.2019.03.08.08.34.24 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 08 Mar 2019 08:34:25 -0800 (PST) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.40 as permitted sender) client-ip=194.138.37.40; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by gecko.sbs.de (8.15.2/8.15.2) with ESMTPS id x28GYO4D016913 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 8 Mar 2019 17:34:24 +0100 Received: from [139.25.68.37] (md1q0hnc.ad001.siemens.net [139.25.68.37] (may be forged)) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id x28GYOKq025538; Fri, 8 Mar 2019 17:34:24 +0100 Subject: Re: [PATCH] Merge patch and unpack tasks To: cedric_hombourger@mentor.com, isar-users References: <9da9415f-28f4-d67d-e1d4-910215591362@siemens.com> <45574f61-229f-442a-b648-81bacc5fdfdb@googlegroups.com> From: Jan Kiszka Message-ID: <978c3d20-9b9e-38fb-9ec2-6a7e3e8cb29c@siemens.com> Date: Fri, 8 Mar 2019 17:34:23 +0100 User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.8.1.12) Gecko/20080226 SUSE/2.0.0.12-1.1 Thunderbird/2.0.0.12 Mnenhy/0.7.5.666 MIME-Version: 1.0 In-Reply-To: <45574f61-229f-442a-b648-81bacc5fdfdb@googlegroups.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-TUID: Not7H+QPM2N3 On 08.03.19 16:58, cedric_hombourger@mentor.com wrote: > Would it be acceptable to have do_patch() use quilt? > If I am not mistaken, do_patch could then start with "quilt pop -a" to remove > all patches from a previous run? > Yes, something along that line. Adds a dependency, but that should be no issue. OE uses quilt or git IIRC, depending on what is patched. I need to check again how hard it would be to port that logic over. Jan > > On Monday, March 4, 2019 at 10:12:01 AM UTC+1, Jan Kiszka wrote: > > On 04.03.19 09:51, [ext] Claudius Heine wrote: > > Hi Jan, > > > > On 23/02/2019 11.44, Jan Kiszka wrote: > >> From: Jan Kiszka > > >> > >> Our patch implementations does not support rolling back previously > >> applied patches, thus fails on patch updates without a rerun of unpack. > >> Avoid this by moving both steps into the same task. > > > > IMO merging multiple conceptual different tasks into one is a step back, > > especially with such a fundamental task as do_patch. > > > > If our implementation lacks behind the one in OE, why not copy stuff from > there > > instead? > > Because it's OE and not Isar. I would have copied things already if they were > well isolated on the OE side. But there seem to be more complex dependencies > that need to be resolved - or we need a conceptually equivalent solution. We > likely need to go that path as we need to account for the debian source package > use case as well. > > Jan > > > > > If that is just impossible for whatever reason, then I would rather bind > those > > tasks together with [postfuncs]. That would allow each function to be > customized > > from other layers if necessary. > > > > regards, > > Claudius > > > >> > >> Signed-off-by: Jan Kiszka > > >> --- > >>   meta/classes/base.bbclass                      | 47 > ++++++++++++++++++---- > >>   meta/classes/dpkg-base.bbclass                     |  9 ++--- > >>   meta/classes/dpkg-raw.bbclass                      |  2 +- > >>   meta/classes/fit-img.bbclass                      |  2 +- > >>   meta/classes/image.bbclass                      |  2 +- > >>   meta/classes/patch.bbclass                      | 33 --------------- > >>   meta/classes/template.bbclass                      |  2 +- > >>   meta/classes/ubi-img.bbclass                      |  2 +- > >>   .../isar-bootstrap/isar-bootstrap-host.bb >           |  2 +- > >>   .../isar-bootstrap/isar-bootstrap-target.bb >         |  2 +- > >>   .../recipes-core/isar-bootstrap/isar-bootstrap.inc |  2 +- > >>   meta/recipes-devtools/isar-apt/isar-apt.bb >          |  2 +- > >>   .../isar-cfg-localepurge/isar-cfg-localepurge.bb >    |  2 +- > >>   13 files changed, 52 insertions(+), 57 deletions(-) > >>   delete mode 100644 meta/classes/patch.bbclass > >> > >> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass > >> index 4279a68..7bdaf37 100644 > >> --- a/meta/classes/base.bbclass > >> +++ b/meta/classes/base.bbclass > >> @@ -1,4 +1,5 @@ > >>   # Copyright (C) 2003  Chris Larson > >> +# Copyright (c) Siemens AG, 2018 > >>   # > >>   # Permission is hereby granted, free of charge, to any person obtaining a > >>   # copy of this software and associated documentation files (the > "Software"), > >> @@ -17,6 +18,8 @@ > >>   # OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, > >>   # ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR > >>   # OTHER DEALINGS IN THE SOFTWARE. > >> +# > >> +# SPDX-License-Identifier: MIT > >> > >>   THISDIR = "${@os.path.dirname(d.getVar('FILE', True))}" > >> > >> @@ -135,25 +138,53 @@ python do_fetch() { > >> > >>   addtask fetch before do_build > >> > >> -do_unpack[dirs] = "${WORKDIR}" > >> -do_unpack[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > >> +do_unpack_and_patch[dirs] = "${WORKDIR}" > >> +do_unpack_and_patch[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > >> > >> -# Unpack package and put it into working directory > >> -python do_unpack() { > >> -    src_uri = (d.getVar('SRC_URI', True) or "").split() > >> -    if len(src_uri) == 0: > >> +# Unpack package, put it into working directory, and apply potential > patches > >> +python do_unpack_and_patch() { > >> +    import subprocess > >> + > >> +    src_uris = (d.getVar('SRC_URI', True) or "").split() > >> +    if len(src_uris) == 0: > >>           return > >> > >>       rootdir = d.getVar('WORKDIR', True) > >> +    src_dir = d.getVar("S", True) > >> > >> +    # unpack src_uris > >>       try: > >> -        fetcher = bb.fetch2.Fetch(src_uri, d) > >> +        fetcher = bb.fetch2.Fetch(src_uris, d) > >>           fetcher.unpack(rootdir) > >>       except bb.fetch2.BBFetchException as e: > >>           raise bb.build.FuncFailed(e) > >> + > >> +    # apply patches > >> +    for src_uri in src_uris: > >> +        try: > >> +            fetcher = bb.fetch2.Fetch([src_uri], d) > >> + > >> +            apply = fetcher.ud[src_uri].parm.get("apply") > >> +            if apply == "no": > >> +                continue > >> + > >> +            basename = fetcher.ud[src_uri].basename or "" > >> +            if not (basename.endswith(".patch") or apply == "yes"): > >> +                continue > >> + > >> +            striplevel = fetcher.ud[src_uri].parm.get("striplevel") or "1" > >> + > >> +            cmd = "patch --no-backup-if-mismatch -p " + striplevel + \ > >> +                  " --directory " + src_dir + \ > >> +                  " --input " + rootdir + '/' + basename > >> +            bb.note(cmd) > >> +            if subprocess.call(cmd, shell=True) != 0: > >> +                bb.fatal("patching failed") > >> +        except bb.fetch2.BBFetchException as e: > >> +            raise bb.build.FuncFailed(e) > >>   } > >> > >> -addtask unpack after do_fetch before do_build > >> +addtask unpack_and_patch after do_fetch before do_build > >> > >>   addtask build > >>   do_build[dirs] = "${TOPDIR}" > >> diff --git a/meta/classes/dpkg-base.bbclass > b/meta/classes/dpkg-base.bbclass > >> index 742b8ad..8ef1f06 100644 > >> --- a/meta/classes/dpkg-base.bbclass > >> +++ b/meta/classes/dpkg-base.bbclass > >> @@ -15,12 +15,9 @@ do_adjust_git() { > >>       fi > >>   } > >> > >> -addtask adjust_git after do_unpack before do_patch > >> +addtask adjust_git after do_unpack_and_patch before do_prepare_build > >>   do_adjust_git[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > >> > >> -inherit patch > >> -addtask patch after do_adjust_git before do_build > >> - > >>   SRC_APT ?= "" > >> > >>   do_apt_fetch[depends] = "buildchroot-target:do_build" > >> @@ -40,7 +37,7 @@ do_apt_fetch() { > >>       dpkg_undo_mounts > >>   } > >> > >> -addtask apt_fetch after do_unpack before do_patch > >> +addtask apt_fetch before do_unpack_and_patch > >>   do_apt_fetch[lockfiles] += "${REPO_ISAR_DIR}/isar.lock" > >>   do_apt_fetch[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > >> > >> @@ -64,7 +61,7 @@ do_prepare_build() { > >>       true > >>   } > >> > >> -addtask prepare_build after do_patch do_transform_template before do_build > >> +addtask prepare_build after do_unpack_and_patch do_transform_template > before > >> do_build > >>   do_prepare_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > >>   # If Isar recipes depend on each other, they typically need the package > >>   # deployed to isar-apt > >> diff --git a/meta/classes/dpkg-raw.bbclass b/meta/classes/dpkg-raw.bbclass > >> index ea03ea4..e63ba1d 100644 > >> --- a/meta/classes/dpkg-raw.bbclass > >> +++ b/meta/classes/dpkg-raw.bbclass > >> @@ -17,7 +17,7 @@ do_install() { > >> > >>   do_install[cleandirs] = "${D}" > >>   do_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > >> -addtask install after do_unpack before do_prepare_build > >> +addtask install after do_unpack_and_patch before do_prepare_build > >> > >>   do_prepare_build[cleandirs] += "${D}/debian" > >>   do_prepare_build() { > >> diff --git a/meta/classes/fit-img.bbclass b/meta/classes/fit-img.bbclass > >> index edca09f..97f1d1a 100644 > >> --- a/meta/classes/fit-img.bbclass > >> +++ b/meta/classes/fit-img.bbclass > >> @@ -29,4 +29,4 @@ do_fit_image() { > >>       sudo chroot ${BUILDCHROOT_DIR} /usr/bin/mkimage ${MKIMAGE_ARGS} \ > >>                   -f '${PP_WORK}/${FIT_IMAGE_SOURCE}' > >> '${PP_DEPLOY}/${FIT_IMAGE_FILE}' > >>   } > >> -addtask fit_image before do_build after do_copy_boot_files > >> do_install_imager_deps do_unpack do_transform_template > >> +addtask fit_image before do_build after do_copy_boot_files > >> do_install_imager_deps do_unpack_and_patch do_transform_template > >> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass > >> index 574fb46..a5952eb 100644 > >> --- a/meta/classes/image.bbclass > >> +++ b/meta/classes/image.bbclass > >> @@ -92,7 +92,7 @@ do_rootfs() { > >>       die "No root filesystem function defined, please implement in your > recipe" > >>   } > >> > >> -addtask rootfs before do_build after do_unpack > >> +addtask rootfs before do_build after do_unpack_and_patch > >>   do_rootfs[deptask] = "do_deploy_deb" > >> > >>   do_mark_rootfs() { > >> diff --git a/meta/classes/patch.bbclass b/meta/classes/patch.bbclass > >> deleted file mode 100644 > >> index 0bc449f..0000000 > >> --- a/meta/classes/patch.bbclass > >> +++ /dev/null > >> @@ -1,33 +0,0 @@ > >> -# This software is a part of ISAR. > >> -# Copyright (c) Siemens AG, 2018 > >> - > >> -python do_patch() { > >> -    import subprocess > >> - > >> -    workdir = d.getVar("WORKDIR", True) + "/" > >> -    src_dir = d.getVar("S", True) > >> - > >> -    for src_uri in (d.getVar("SRC_URI", True) or "").split(): > >> -        try: > >> -            fetcher = bb.fetch2.Fetch([src_uri], d) > >> - > >> -            apply = fetcher.ud[src_uri].parm.get("apply") > >> -            if apply == "no": > >> -                continue > >> - > >> -            basename = fetcher.ud[src_uri].basename or "" > >> -            if not (basename.endswith(".patch") or apply == "yes"): > >> -                continue > >> - > >> -            striplevel = fetcher.ud[src_uri].parm.get("striplevel") or "1" > >> - > >> -            cmd = "patch --no-backup-if-mismatch -p " + striplevel + \ > >> -                  " --directory " + src_dir + " --input " + workdir + > basename > >> -            bb.note(cmd) > >> -            if subprocess.call(cmd, shell=True) != 0: > >> -                bb.fatal("patching failed") > >> -        except bb.fetch2.BBFetchException as e: > >> -            raise bb.build.FuncFailed(e) > >> -} > >> - > >> -do_patch[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" > >> diff --git a/meta/classes/template.bbclass b/meta/classes/template.bbclass > >> index 324511a..8846ab0 100644 > >> --- a/meta/classes/template.bbclass > >> +++ b/meta/classes/template.bbclass > >> @@ -59,4 +59,4 @@ python do_transform_template() { > >>               if process.wait() != 0: > >>                   bb.fatal("processing of template failed") > >>   } > >> -addtask do_transform_template after do_unpack > >> +addtask do_transform_template after do_unpack_and_patch > >> diff --git a/meta/classes/ubi-img.bbclass b/meta/classes/ubi-img.bbclass > >> index f61a940..f06ab04 100644 > >> --- a/meta/classes/ubi-img.bbclass > >> +++ b/meta/classes/ubi-img.bbclass > >> @@ -32,4 +32,4 @@ do_ubi_image() { > >>       sudo chroot ${BUILDCHROOT_DIR} /usr/sbin/ubinize ${UBINIZE_ARGS} \ > >>                   -o '${PP_DEPLOY}/${UBI_IMAGE_FILE}' > '${PP_WORK}/${UBINIZE_CFG}' > >>   } > >> -addtask ubi_image before do_build after do_copy_boot_files > >> do_install_imager_deps do_unpack do_transform_template > >> +addtask ubi_image before do_build after do_copy_boot_files > >> do_install_imager_deps do_unpack_and_patch do_transform_template > >> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb > > >> b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb > > >> index a793585..da6d5e1 100644 > >> --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb > > >> +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb > > >> @@ -40,7 +40,7 @@ python do_apt_config_prepare() { > >>           aggregate_files(d, apt_sources_list, apt_sources_init_out) > >>           aggregate_aptsources_list(d, apt_sources_list, apt_sources_out) > >>   } > >> -addtask apt_config_prepare before do_bootstrap after do_unpack > >> +addtask apt_config_prepare before do_bootstrap after do_unpack_and_patch > >> > >>   OVERRIDES_append = ":${@get_distro_needs_https_support(d, True)}" > >> > >> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb > > >> b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb > > >> index bec6fa8..b338adf 100644 > >> --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb > > >> +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb > > >> @@ -39,7 +39,7 @@ python do_apt_config_prepare() { > >>           aggregate_files(d, apt_sources_list, apt_sources_init_out) > >>           aggregate_aptsources_list(d, apt_sources_list, apt_sources_out) > >>   } > >> -addtask apt_config_prepare before do_bootstrap after do_unpack > >> +addtask apt_config_prepare before do_bootstrap after do_unpack_and_patch > >> > >>   OVERRIDES_append = ":${@get_distro_needs_https_support(d, False)}" > >> > >> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > >> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > >> index 234d339..15995d4 100644 > >> --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > >> +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc > >> @@ -184,7 +184,7 @@ do_generate_keyring() { > >>           done > >>       fi > >>   } > >> -addtask generate_keyring before do_build after do_unpack > >> +addtask generate_keyring before do_build after do_unpack_and_patch > >> > >> > >> > >> diff --git a/meta/recipes-devtools/isar-apt/isar-apt.bb > > >> b/meta/recipes-devtools/isar-apt/isar-apt.bb > >> index a959691..aea7ff7 100644 > >> --- a/meta/recipes-devtools/isar-apt/isar-apt.bb > >> +++ b/meta/recipes-devtools/isar-apt/isar-apt.bb > >> @@ -26,4 +26,4 @@ do_cache_config() { > >>       fi > >>   } > >> > >> -addtask cache_config after do_unpack before do_build > >> +addtask cache_config after do_unpack_and_patch before do_build > >> diff --git > a/meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb > > >> b/meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb > > >> index 62b4b2d..b231e1b 100644 > >> --- a/meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb > > >> +++ b/meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb > > >> @@ -55,7 +55,7 @@ do_gen_config() { > >>           ${@get_nopurge(d)} > >>       __EOF__ > >>   } > >> -addtask gen_config after do_unpack before do_install > >> +addtask gen_config after do_unpack_and_patch before do_install > >> > >>   do_install() { > >>       install -v -d ${D}/usr/lib/${PN} > >> -- > >> 2.16.4 > >> > > -- > Siemens AG, Corporate Technology, CT RDA IOT SES-DE > Corporate Competence Center Embedded Linux > > -- > You received this message because you are subscribed to the Google Groups > "isar-users" group. > To unsubscribe from this group and stop receiving emails from it, send an email > to isar-users+unsubscribe@googlegroups.com > . > To post to this group, send email to isar-users@googlegroups.com > . > To view this discussion on the web visit > https://groups.google.com/d/msgid/isar-users/45574f61-229f-442a-b648-81bacc5fdfdb%40googlegroups.com > . > For more options, visit https://groups.google.com/d/optout. -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux