From: Jan Kiszka <jan.kiszka@siemens.com>
To: cedric_hombourger@mentor.com, isar-users <isar-users@googlegroups.com>
Subject: Re: [PATCH] Merge patch and unpack tasks
Date: Fri, 8 Mar 2019 17:34:23 +0100 [thread overview]
Message-ID: <978c3d20-9b9e-38fb-9ec2-6a7e3e8cb29c@siemens.com> (raw)
In-Reply-To: <45574f61-229f-442a-b648-81bacc5fdfdb@googlegroups.com>
On 08.03.19 16:58, cedric_hombourger@mentor.com wrote:
> Would it be acceptable to have do_patch() use quilt?
> If I am not mistaken, do_patch could then start with "quilt pop -a" to remove
> all patches from a previous run?
>
Yes, something along that line. Adds a dependency, but that should be no issue.
OE uses quilt or git IIRC, depending on what is patched. I need to check again
how hard it would be to port that logic over.
Jan
>
> On Monday, March 4, 2019 at 10:12:01 AM UTC+1, Jan Kiszka wrote:
>
> On 04.03.19 09:51, [ext] Claudius Heine wrote:
> > Hi Jan,
> >
> > On 23/02/2019 11.44, Jan Kiszka wrote:
> >> From: Jan Kiszka <jan.k...@siemens.com <javascript:>>
> >>
> >> Our patch implementations does not support rolling back previously
> >> applied patches, thus fails on patch updates without a rerun of unpack.
> >> Avoid this by moving both steps into the same task.
> >
> > IMO merging multiple conceptual different tasks into one is a step back,
> > especially with such a fundamental task as do_patch.
> >
> > If our implementation lacks behind the one in OE, why not copy stuff from
> there
> > instead?
>
> Because it's OE and not Isar. I would have copied things already if they were
> well isolated on the OE side. But there seem to be more complex dependencies
> that need to be resolved - or we need a conceptually equivalent solution. We
> likely need to go that path as we need to account for the debian source package
> use case as well.
>
> Jan
>
> >
> > If that is just impossible for whatever reason, then I would rather bind
> those
> > tasks together with [postfuncs]. That would allow each function to be
> customized
> > from other layers if necessary.
> >
> > regards,
> > Claudius
> >
> >>
> >> Signed-off-by: Jan Kiszka <jan.k...@siemens.com <javascript:>>
> >> ---
> >> meta/classes/base.bbclass | 47
> ++++++++++++++++++----
> >> meta/classes/dpkg-base.bbclass | 9 ++---
> >> meta/classes/dpkg-raw.bbclass | 2 +-
> >> meta/classes/fit-img.bbclass | 2 +-
> >> meta/classes/image.bbclass | 2 +-
> >> meta/classes/patch.bbclass | 33 ---------------
> >> meta/classes/template.bbclass | 2 +-
> >> meta/classes/ubi-img.bbclass | 2 +-
> >> .../isar-bootstrap/isar-bootstrap-host.bb
> <http://isar-bootstrap-host.bb> | 2 +-
> >> .../isar-bootstrap/isar-bootstrap-target.bb
> <http://isar-bootstrap-target.bb> | 2 +-
> >> .../recipes-core/isar-bootstrap/isar-bootstrap.inc | 2 +-
> >> meta/recipes-devtools/isar-apt/isar-apt.bb
> <http://isar-apt.bb> | 2 +-
> >> .../isar-cfg-localepurge/isar-cfg-localepurge.bb
> <http://isar-cfg-localepurge.bb> | 2 +-
> >> 13 files changed, 52 insertions(+), 57 deletions(-)
> >> delete mode 100644 meta/classes/patch.bbclass
> >>
> >> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> >> index 4279a68..7bdaf37 100644
> >> --- a/meta/classes/base.bbclass
> >> +++ b/meta/classes/base.bbclass
> >> @@ -1,4 +1,5 @@
> >> # Copyright (C) 2003 Chris Larson
> >> +# Copyright (c) Siemens AG, 2018
> >> #
> >> # Permission is hereby granted, free of charge, to any person obtaining a
> >> # copy of this software and associated documentation files (the
> "Software"),
> >> @@ -17,6 +18,8 @@
> >> # OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE,
> >> # ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
> >> # OTHER DEALINGS IN THE SOFTWARE.
> >> +#
> >> +# SPDX-License-Identifier: MIT
> >>
> >> THISDIR = "${@os.path.dirname(d.getVar('FILE', True))}"
> >>
> >> @@ -135,25 +138,53 @@ python do_fetch() {
> >>
> >> addtask fetch before do_build
> >>
> >> -do_unpack[dirs] = "${WORKDIR}"
> >> -do_unpack[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> >> +do_unpack_and_patch[dirs] = "${WORKDIR}"
> >> +do_unpack_and_patch[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> >>
> >> -# Unpack package and put it into working directory
> >> -python do_unpack() {
> >> - src_uri = (d.getVar('SRC_URI', True) or "").split()
> >> - if len(src_uri) == 0:
> >> +# Unpack package, put it into working directory, and apply potential
> patches
> >> +python do_unpack_and_patch() {
> >> + import subprocess
> >> +
> >> + src_uris = (d.getVar('SRC_URI', True) or "").split()
> >> + if len(src_uris) == 0:
> >> return
> >>
> >> rootdir = d.getVar('WORKDIR', True)
> >> + src_dir = d.getVar("S", True)
> >>
> >> + # unpack src_uris
> >> try:
> >> - fetcher = bb.fetch2.Fetch(src_uri, d)
> >> + fetcher = bb.fetch2.Fetch(src_uris, d)
> >> fetcher.unpack(rootdir)
> >> except bb.fetch2.BBFetchException as e:
> >> raise bb.build.FuncFailed(e)
> >> +
> >> + # apply patches
> >> + for src_uri in src_uris:
> >> + try:
> >> + fetcher = bb.fetch2.Fetch([src_uri], d)
> >> +
> >> + apply = fetcher.ud[src_uri].parm.get("apply")
> >> + if apply == "no":
> >> + continue
> >> +
> >> + basename = fetcher.ud[src_uri].basename or ""
> >> + if not (basename.endswith(".patch") or apply == "yes"):
> >> + continue
> >> +
> >> + striplevel = fetcher.ud[src_uri].parm.get("striplevel") or "1"
> >> +
> >> + cmd = "patch --no-backup-if-mismatch -p " + striplevel + \
> >> + " --directory " + src_dir + \
> >> + " --input " + rootdir + '/' + basename
> >> + bb.note(cmd)
> >> + if subprocess.call(cmd, shell=True) != 0:
> >> + bb.fatal("patching failed")
> >> + except bb.fetch2.BBFetchException as e:
> >> + raise bb.build.FuncFailed(e)
> >> }
> >>
> >> -addtask unpack after do_fetch before do_build
> >> +addtask unpack_and_patch after do_fetch before do_build
> >>
> >> addtask build
> >> do_build[dirs] = "${TOPDIR}"
> >> diff --git a/meta/classes/dpkg-base.bbclass
> b/meta/classes/dpkg-base.bbclass
> >> index 742b8ad..8ef1f06 100644
> >> --- a/meta/classes/dpkg-base.bbclass
> >> +++ b/meta/classes/dpkg-base.bbclass
> >> @@ -15,12 +15,9 @@ do_adjust_git() {
> >> fi
> >> }
> >>
> >> -addtask adjust_git after do_unpack before do_patch
> >> +addtask adjust_git after do_unpack_and_patch before do_prepare_build
> >> do_adjust_git[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> >>
> >> -inherit patch
> >> -addtask patch after do_adjust_git before do_build
> >> -
> >> SRC_APT ?= ""
> >>
> >> do_apt_fetch[depends] = "buildchroot-target:do_build"
> >> @@ -40,7 +37,7 @@ do_apt_fetch() {
> >> dpkg_undo_mounts
> >> }
> >>
> >> -addtask apt_fetch after do_unpack before do_patch
> >> +addtask apt_fetch before do_unpack_and_patch
> >> do_apt_fetch[lockfiles] += "${REPO_ISAR_DIR}/isar.lock"
> >> do_apt_fetch[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> >>
> >> @@ -64,7 +61,7 @@ do_prepare_build() {
> >> true
> >> }
> >>
> >> -addtask prepare_build after do_patch do_transform_template before do_build
> >> +addtask prepare_build after do_unpack_and_patch do_transform_template
> before
> >> do_build
> >> do_prepare_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> >> # If Isar recipes depend on each other, they typically need the package
> >> # deployed to isar-apt
> >> diff --git a/meta/classes/dpkg-raw.bbclass b/meta/classes/dpkg-raw.bbclass
> >> index ea03ea4..e63ba1d 100644
> >> --- a/meta/classes/dpkg-raw.bbclass
> >> +++ b/meta/classes/dpkg-raw.bbclass
> >> @@ -17,7 +17,7 @@ do_install() {
> >>
> >> do_install[cleandirs] = "${D}"
> >> do_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> >> -addtask install after do_unpack before do_prepare_build
> >> +addtask install after do_unpack_and_patch before do_prepare_build
> >>
> >> do_prepare_build[cleandirs] += "${D}/debian"
> >> do_prepare_build() {
> >> diff --git a/meta/classes/fit-img.bbclass b/meta/classes/fit-img.bbclass
> >> index edca09f..97f1d1a 100644
> >> --- a/meta/classes/fit-img.bbclass
> >> +++ b/meta/classes/fit-img.bbclass
> >> @@ -29,4 +29,4 @@ do_fit_image() {
> >> sudo chroot ${BUILDCHROOT_DIR} /usr/bin/mkimage ${MKIMAGE_ARGS} \
> >> -f '${PP_WORK}/${FIT_IMAGE_SOURCE}'
> >> '${PP_DEPLOY}/${FIT_IMAGE_FILE}'
> >> }
> >> -addtask fit_image before do_build after do_copy_boot_files
> >> do_install_imager_deps do_unpack do_transform_template
> >> +addtask fit_image before do_build after do_copy_boot_files
> >> do_install_imager_deps do_unpack_and_patch do_transform_template
> >> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> >> index 574fb46..a5952eb 100644
> >> --- a/meta/classes/image.bbclass
> >> +++ b/meta/classes/image.bbclass
> >> @@ -92,7 +92,7 @@ do_rootfs() {
> >> die "No root filesystem function defined, please implement in your
> recipe"
> >> }
> >>
> >> -addtask rootfs before do_build after do_unpack
> >> +addtask rootfs before do_build after do_unpack_and_patch
> >> do_rootfs[deptask] = "do_deploy_deb"
> >>
> >> do_mark_rootfs() {
> >> diff --git a/meta/classes/patch.bbclass b/meta/classes/patch.bbclass
> >> deleted file mode 100644
> >> index 0bc449f..0000000
> >> --- a/meta/classes/patch.bbclass
> >> +++ /dev/null
> >> @@ -1,33 +0,0 @@
> >> -# This software is a part of ISAR.
> >> -# Copyright (c) Siemens AG, 2018
> >> -
> >> -python do_patch() {
> >> - import subprocess
> >> -
> >> - workdir = d.getVar("WORKDIR", True) + "/"
> >> - src_dir = d.getVar("S", True)
> >> -
> >> - for src_uri in (d.getVar("SRC_URI", True) or "").split():
> >> - try:
> >> - fetcher = bb.fetch2.Fetch([src_uri], d)
> >> -
> >> - apply = fetcher.ud[src_uri].parm.get("apply")
> >> - if apply == "no":
> >> - continue
> >> -
> >> - basename = fetcher.ud[src_uri].basename or ""
> >> - if not (basename.endswith(".patch") or apply == "yes"):
> >> - continue
> >> -
> >> - striplevel = fetcher.ud[src_uri].parm.get("striplevel") or "1"
> >> -
> >> - cmd = "patch --no-backup-if-mismatch -p " + striplevel + \
> >> - " --directory " + src_dir + " --input " + workdir +
> basename
> >> - bb.note(cmd)
> >> - if subprocess.call(cmd, shell=True) != 0:
> >> - bb.fatal("patching failed")
> >> - except bb.fetch2.BBFetchException as e:
> >> - raise bb.build.FuncFailed(e)
> >> -}
> >> -
> >> -do_patch[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> >> diff --git a/meta/classes/template.bbclass b/meta/classes/template.bbclass
> >> index 324511a..8846ab0 100644
> >> --- a/meta/classes/template.bbclass
> >> +++ b/meta/classes/template.bbclass
> >> @@ -59,4 +59,4 @@ python do_transform_template() {
> >> if process.wait() != 0:
> >> bb.fatal("processing of template failed")
> >> }
> >> -addtask do_transform_template after do_unpack
> >> +addtask do_transform_template after do_unpack_and_patch
> >> diff --git a/meta/classes/ubi-img.bbclass b/meta/classes/ubi-img.bbclass
> >> index f61a940..f06ab04 100644
> >> --- a/meta/classes/ubi-img.bbclass
> >> +++ b/meta/classes/ubi-img.bbclass
> >> @@ -32,4 +32,4 @@ do_ubi_image() {
> >> sudo chroot ${BUILDCHROOT_DIR} /usr/sbin/ubinize ${UBINIZE_ARGS} \
> >> -o '${PP_DEPLOY}/${UBI_IMAGE_FILE}'
> '${PP_WORK}/${UBINIZE_CFG}'
> >> }
> >> -addtask ubi_image before do_build after do_copy_boot_files
> >> do_install_imager_deps do_unpack do_transform_template
> >> +addtask ubi_image before do_build after do_copy_boot_files
> >> do_install_imager_deps do_unpack_and_patch do_transform_template
> >> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
> <http://isar-bootstrap-host.bb>
> >> b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
> <http://isar-bootstrap-host.bb>
> >> index a793585..da6d5e1 100644
> >> --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
> <http://isar-bootstrap-host.bb>
> >> +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-host.bb
> <http://isar-bootstrap-host.bb>
> >> @@ -40,7 +40,7 @@ python do_apt_config_prepare() {
> >> aggregate_files(d, apt_sources_list, apt_sources_init_out)
> >> aggregate_aptsources_list(d, apt_sources_list, apt_sources_out)
> >> }
> >> -addtask apt_config_prepare before do_bootstrap after do_unpack
> >> +addtask apt_config_prepare before do_bootstrap after do_unpack_and_patch
> >>
> >> OVERRIDES_append = ":${@get_distro_needs_https_support(d, True)}"
> >>
> >> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb
> <http://isar-bootstrap-target.bb>
> >> b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb
> <http://isar-bootstrap-target.bb>
> >> index bec6fa8..b338adf 100644
> >> --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb
> <http://isar-bootstrap-target.bb>
> >> +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap-target.bb
> <http://isar-bootstrap-target.bb>
> >> @@ -39,7 +39,7 @@ python do_apt_config_prepare() {
> >> aggregate_files(d, apt_sources_list, apt_sources_init_out)
> >> aggregate_aptsources_list(d, apt_sources_list, apt_sources_out)
> >> }
> >> -addtask apt_config_prepare before do_bootstrap after do_unpack
> >> +addtask apt_config_prepare before do_bootstrap after do_unpack_and_patch
> >>
> >> OVERRIDES_append = ":${@get_distro_needs_https_support(d, False)}"
> >>
> >> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> >> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> >> index 234d339..15995d4 100644
> >> --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> >> +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> >> @@ -184,7 +184,7 @@ do_generate_keyring() {
> >> done
> >> fi
> >> }
> >> -addtask generate_keyring before do_build after do_unpack
> >> +addtask generate_keyring before do_build after do_unpack_and_patch
> >>
> >>
> >>
> >> diff --git a/meta/recipes-devtools/isar-apt/isar-apt.bb
> <http://isar-apt.bb>
> >> b/meta/recipes-devtools/isar-apt/isar-apt.bb <http://isar-apt.bb>
> >> index a959691..aea7ff7 100644
> >> --- a/meta/recipes-devtools/isar-apt/isar-apt.bb <http://isar-apt.bb>
> >> +++ b/meta/recipes-devtools/isar-apt/isar-apt.bb <http://isar-apt.bb>
> >> @@ -26,4 +26,4 @@ do_cache_config() {
> >> fi
> >> }
> >>
> >> -addtask cache_config after do_unpack before do_build
> >> +addtask cache_config after do_unpack_and_patch before do_build
> >> diff --git
> a/meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb
> <http://isar-cfg-localepurge.bb>
> >> b/meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb
> <http://isar-cfg-localepurge.bb>
> >> index 62b4b2d..b231e1b 100644
> >> --- a/meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb
> <http://isar-cfg-localepurge.bb>
> >> +++ b/meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb
> <http://isar-cfg-localepurge.bb>
> >> @@ -55,7 +55,7 @@ do_gen_config() {
> >> ${@get_nopurge(d)}
> >> __EOF__
> >> }
> >> -addtask gen_config after do_unpack before do_install
> >> +addtask gen_config after do_unpack_and_patch before do_install
> >>
> >> do_install() {
> >> install -v -d ${D}/usr/lib/${PN}
> >> --
> >> 2.16.4
> >>
>
> --
> Siemens AG, Corporate Technology, CT RDA IOT SES-DE
> Corporate Competence Center Embedded Linux
>
> --
> You received this message because you are subscribed to the Google Groups
> "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send an email
> to isar-users+unsubscribe@googlegroups.com
> <mailto:isar-users+unsubscribe@googlegroups.com>.
> To post to this group, send email to isar-users@googlegroups.com
> <mailto:isar-users@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/isar-users/45574f61-229f-442a-b648-81bacc5fdfdb%40googlegroups.com
> <https://groups.google.com/d/msgid/isar-users/45574f61-229f-442a-b648-81bacc5fdfdb%40googlegroups.com?utm_medium=email&utm_source=footer>.
> For more options, visit https://groups.google.com/d/optout.
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
prev parent reply other threads:[~2019-03-08 16:34 UTC|newest]
Thread overview: 7+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-02-23 10:44 Jan Kiszka
2019-02-25 9:47 ` Henning Schild
2019-02-25 10:01 ` Jan Kiszka
2019-03-04 8:51 ` Claudius Heine
2019-03-04 9:11 ` Jan Kiszka
2019-03-08 15:58 ` cedric_hombourger
2019-03-08 16:34 ` Jan Kiszka [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=978c3d20-9b9e-38fb-9ec2-6a7e3e8cb29c@siemens.com \
--to=jan.kiszka@siemens.com \
--cc=cedric_hombourger@mentor.com \
--cc=isar-users@googlegroups.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox