[PATCH 0/4] Patch improvements, more templating fallouts, linux-libc-dev fix

[PATCH 0/4] Patch improvements, more templating fallouts, linux-libc-dev fix

[Jan Kiszka]

See patches for details.

Jan Kiszka (4):
  patch: Add support for patchdir parameter
  Make patch task repeatedly applicable
  template: Remove KERNEL_NAME from default TEMPLATE_VARS
  linux-custom: Fix linux-libc-dev version check

 meta/classes/patch.bbclass                      | 57 ++++++++++++++++++++++---
 meta/classes/template.bbclass                   |  2 +-
 meta/recipes-kernel/linux-module/module.inc     |  1 +
 meta/recipes-kernel/linux/files/build-kernel.sh |  2 +-
 4 files changed, 53 insertions(+), 9 deletions(-)

-- 
2.16.4

[PATCH 1/4] patch: Add support for patchdir parameter

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

This allows to define the target directory of a patch, just like OE
supports it. Default remains ${S}.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 meta/classes/patch.bbclass | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/classes/patch.bbclass b/meta/classes/patch.bbclass
index c19542d..4c08193 100644
--- a/meta/classes/patch.bbclass
+++ b/meta/classes/patch.bbclass
@@ -19,13 +19,14 @@ python do_patch() {
             if not (basename.endswith(".patch") or apply == "yes"):
                 continue
 
+            patchdir = fetcher.ud[src_uri].parm.get("patchdir") or src_dir
             striplevel = fetcher.ud[src_uri].parm.get("striplevel") or "1"
 
             cmd = [
                 "patch",
                 "--no-backup-if-mismatch",
                 "-p", striplevel,
-                "--directory", src_dir,
+                "--directory", patchdir,
                 "--input", workdir + basename,
             ]
             bb.note(" ".join(cmd))
-- 
2.16.4

[PATCH 2/4] Make patch task repeatedly applicable

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

This allows to rerun the patch task even if it was applied already. This
is achieved by recording the executed patch commands and storing them,
along with the original patches, in ${S}/.applied_patches and applying
them in reverse order before applying new patches. If unpack ran prior
to that, .applied_patches was deleted, and nothing is incorrectly
reverted.

Alternatives to this self-written solution would have been using quilt,
which would have added almost 30 MB additional dependencies to our build
environment, or even using the OE implementation for the patch task,
which would have required importing and adjusting a significant amount
of code with only little gain.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 meta/classes/patch.bbclass | 54 ++++++++++++++++++++++++++++++++++++++++------
 1 file changed, 48 insertions(+), 6 deletions(-)

diff --git a/meta/classes/patch.bbclass b/meta/classes/patch.bbclass
index 4c08193..bb203b7 100644
--- a/meta/classes/patch.bbclass
+++ b/meta/classes/patch.bbclass
@@ -1,13 +1,38 @@
 # This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2018
+# Copyright (c) Siemens AG, 2018-2019
+
+def clean_applied_patches(applied_patches_dir):
+    import shutil
+    import subprocess
+
+    if not os.path.exists(applied_patches_dir):
+        return
+
+    cmds_file = applied_patches_dir + ".patch-commands"
+    if os.path.exists(cmds_file):
+        with open(cmds_file, "r") as cmds:
+            patch_commands = cmds.readlines()
+        patch_commands.reverse()
+        for patch in patch_commands:
+            cmd = patch.split()
+            cmd.append("-R")
+            cmdline = " ".join(cmd)
+            bb.note("Reverting: " + cmdline)
+            if subprocess.call(cmd) != 0:
+                bb.fatal("patch reverting failed")
+
+    shutil.rmtree(applied_patches_dir)
 
 python do_patch() {
+    import shutil
     import subprocess
 
-    workdir = d.getVar("WORKDIR", True) + "/"
-    src_dir = d.getVar("S", True)
+    workdir = d.getVar("WORKDIR") + "/"
+    src_dir = d.getVar("S")
+
+    applied_patches_dirs = []
 
-    for src_uri in (d.getVar("SRC_URI", True) or "").split():
+    for src_uri in (d.getVar("SRC_URI") or "").split():
         try:
             fetcher = bb.fetch2.Fetch([src_uri], d)
 
@@ -20,6 +45,18 @@ python do_patch() {
                 continue
 
             patchdir = fetcher.ud[src_uri].parm.get("patchdir") or src_dir
+            applied_patches_dir = patchdir + "/.applied_patches/"
+
+            if applied_patches_dir not in applied_patches_dirs:
+                clean_applied_patches(applied_patches_dir)
+                bb.utils.mkdirhier(applied_patches_dir)
+                applied_patches_dirs.append(applied_patches_dir)
+
+            cmds = open(applied_patches_dir + ".patch-commands", "a")
+
+            patch_file = applied_patches_dir + basename
+            shutil.copyfile(workdir + basename, patch_file)
+
             striplevel = fetcher.ud[src_uri].parm.get("striplevel") or "1"
 
             cmd = [
@@ -27,11 +64,16 @@ python do_patch() {
                 "--no-backup-if-mismatch",
                 "-p", striplevel,
                 "--directory", patchdir,
-                "--input", workdir + basename,
+                "--input", patch_file,
             ]
-            bb.note(" ".join(cmd))
+            cmdline = " ".join(cmd)
+
+            bb.note("Applying: " + cmdline)
             if subprocess.call(cmd) != 0:
                 bb.fatal("patching failed")
+
+            cmds.write(cmdline + "\n")
+            cmds.close()
         except bb.fetch2.BBFetchException as e:
             raise bb.build.FuncFailed(e)
 }
-- 
2.16.4

[PATCH 3/4] template: Remove KERNEL_NAME from default TEMPLATE_VARS

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

KERNEL_NAME is a globally defined variable but is only used for
templating by the module recipe include. As the templating step is
inherited by all packages, changing the KERNEL_NAME can cause unneeded
recipe rebuilds. Therefore, make this template variable specific.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 meta/classes/template.bbclass               | 2 +-
 meta/recipes-kernel/linux-module/module.inc | 1 +
 2 files changed, 2 insertions(+), 1 deletion(-)

diff --git a/meta/classes/template.bbclass b/meta/classes/template.bbclass
index 3eb9114..f3b5f80 100644
--- a/meta/classes/template.bbclass
+++ b/meta/classes/template.bbclass
@@ -4,7 +4,7 @@
 # SPDX-License-Identifier: MIT
 
 TEMPLATE_FILES ?= ""
-TEMPLATE_VARS ?= "PN PV DESCRIPTION HOMEPAGE MAINTAINER KERNEL_NAME DISTRO_ARCH"
+TEMPLATE_VARS ?= "PN PV DESCRIPTION HOMEPAGE MAINTAINER DISTRO_ARCH"
 
 do_transform_template[vardeps] = "TEMPLATE_FILES ${TEMPLATE_VARS}"
 do_transform_template[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
diff --git a/meta/recipes-kernel/linux-module/module.inc b/meta/recipes-kernel/linux-module/module.inc
index 44edd58..6506e39 100644
--- a/meta/recipes-kernel/linux-module/module.inc
+++ b/meta/recipes-kernel/linux-module/module.inc
@@ -21,6 +21,7 @@ inherit dpkg
 
 TEMPLATE_FILES = "debian/control.tmpl \
                   debian/changelog.tmpl"
+TEMPLATE_VARS += "KERNEL_NAME"
 
 do_prepare_build() {
     cp -r ${WORKDIR}/debian ${S}/
-- 
2.16.4

[PATCH 4/4] linux-custom: Fix linux-libc-dev version check

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

We deploy "${PV}-1". Comparing against "${PV}" prevents package updates
and triggers false-positive warnings.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 meta/recipes-kernel/linux/files/build-kernel.sh | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh
index 842b585..4fdf3ed 100644
--- a/meta/recipes-kernel/linux/files/build-kernel.sh
+++ b/meta/recipes-kernel/linux/files/build-kernel.sh
@@ -127,6 +127,6 @@ rm -f linux-headers-${PV}_${PV}-1_*.deb
 # linux-libc-dev causes dependency problems if we downgrade
 # remove it after the build so the downgraded version does not get deployed
 LINUX_LIBC_DEV_V=$( dpkg-query --show --showformat '${Version}' linux-libc-dev )
-if dpkg --compare-versions $LINUX_LIBC_DEV_V gt $PV; then
+if dpkg --compare-versions $LINUX_LIBC_DEV_V gt $PV-1; then
 	rm -f linux-libc-dev_${PV}*.deb
 fi
-- 
2.16.4

Re: [PATCH 0/4] Patch improvements, more templating fallouts, linux-libc-dev fix

[Maxim Yu. Osipov]

On 4/2/19 8:20 AM, Jan Kiszka wrote:
> See patches for details.

Applied to the 'next',

Thanks,
Maxim.

> Jan Kiszka (4):
>    patch: Add support for patchdir parameter
>    Make patch task repeatedly applicable
>    template: Remove KERNEL_NAME from default TEMPLATE_VARS
>    linux-custom: Fix linux-libc-dev version check
> 
>   meta/classes/patch.bbclass                      | 57 ++++++++++++++++++++++---
>   meta/classes/template.bbclass                   |  2 +-
>   meta/recipes-kernel/linux-module/module.inc     |  1 +
>   meta/recipes-kernel/linux/files/build-kernel.sh |  2 +-
>   4 files changed, 53 insertions(+), 9 deletions(-)
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov