[PATCH v3 0/2] Custom ${S} series

[PATCH v3 0/2] Custom ${S} series

[Vijai Kumar K]

Changes since V2:
- Introduce P2 to demonstrate apt:// without ${PV} set.
- Address review comment from Henning.
- Fix issue where do_apt_unpack is triggered for non apt://
  SRC_URIs.

CI build going on: http://ci.isar-build.org:8080/job/isar_vkk_devel/93/

Changes since V1:

- Introduce a separate do_apt_unpack task

Vijai Kumar K (2):
  dpkg-base: Introduce do_apt_unpack
  Modify hello to demonstrate apt:// without ${PV}

 .../recipes-app/hello/{hello.inc => hello.bb} |  0
 meta-isar/recipes-app/hello/hello_2.10.bb     | 11 -------
 meta-isar/recipes-app/hello/hello_2.9.bb      |  8 -----
 meta/classes/dpkg-base.bbclass                | 29 ++++++++++++++++---
 4 files changed, 25 insertions(+), 23 deletions(-)
 rename meta-isar/recipes-app/hello/{hello.inc => hello.bb} (100%)
 delete mode 100644 meta-isar/recipes-app/hello/hello_2.10.bb
 delete mode 100644 meta-isar/recipes-app/hello/hello_2.9.bb

-- 
2.17.1

[PATCH v3 1/2] dpkg-base: Introduce do_apt_unpack

[Vijai Kumar K]

The Debian source package fetch and unpacking happens now inside the
do_apt_fetch task.

With the current do_apt_fetch implementation, it is not possible to use
a custom source directory(${S}).
apt-get source by default extracts the contents of the debian source
into folder with name <pkg>_<version>.

Add provision for specifying a custom source directory.

Add a new task called do_apt_unpack and move unpacking logic there.

Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
---
 meta/classes/dpkg-base.bbclass | 29 +++++++++++++++++++++++++----
 1 file changed, 25 insertions(+), 4 deletions(-)

diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index 5c7bddc..c1425bd 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -58,7 +58,6 @@ do_apt_fetch() {
     if [ -z "${@d.getVar("SRC_APT", True).strip()}" ]; then
         return 0
     fi
-    rm -rf ${S}
     dpkg_do_mounts
     E="${@ isar_export_proxies(d)}"
     sudo -E chroot ${BUILDCHROOT_DIR} /usr/bin/apt-get update \
@@ -69,16 +68,38 @@ do_apt_fetch() {
     for uri in "${SRC_APT}"; do
         sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \
             sh -c 'mkdir -p /downloads/deb-src/"$1"/"$2" && cd /downloads/deb-src/"$1"/"$2" && apt-get -y --download-only --only-source source "$2"' my_script "${DISTRO}" "${uri}"
-        sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \
-            sh -c 'cp /downloads/deb-src/"$1"/"$2"/* ${PP} && cd ${PP} && apt-get -y --only-source source "$2"' my_script "${DISTRO}" "${uri}"
     done
 
     dpkg_undo_mounts
 }
 
-addtask apt_fetch after do_unpack before do_patch
+addtask apt_fetch after do_unpack before do_apt_unpack
 do_apt_fetch[lockfiles] += "${REPO_ISAR_DIR}/isar.lock"
 
+do_apt_unpack() {
+    if [ -z "${@d.getVar("SRC_APT", True).strip()}" ]; then
+        return 0
+    fi
+    rm -rf ${S}
+    dpkg_do_mounts
+    E="${@ isar_export_proxies(d)}"
+
+    for uri in "${SRC_APT}"; do
+        sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \
+            sh -c ' \
+                set -e
+                dscfile="$(apt-get -y -qq --print-uris source "${2}" | cut -d " " -f2 | grep -E "*.dsc")"
+                cd ${PP}
+                cp /downloads/deb-src/"${1}"/"${2}"/* ${PP}
+                dpkg-source -x "${dscfile}" "${PPS}"' \
+                    my_script "${DISTRO}" "${uri}"
+    done
+
+    dpkg_undo_mounts
+}
+
+addtask apt_unpack after do_apt_fetch before do_patch
+
 addtask cleanall_apt before do_cleanall
 do_cleanall_apt[nostamp] = "1"
 do_cleanall_apt() {
-- 
2.17.1

[PATCH v3 2/2] Modify hello to demonstrate apt:// without ${PV}

[Vijai Kumar K]

With the introduction of do_apt_unpack, we can set custom ${S}.

With that feature in place it is no longer mandatory to specify
the ${PV} of the package.

${PV} was mandatory before because do_apt_fetch was unpacking
the contents into a folder named "<debian_source>-<debian-source-version>"
hardcoding ${S} to it. So we needed to know the correct
${PV}(debian-source-version) beforehand.

Signed-off-by: Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>
---
 meta-isar/recipes-app/hello/{hello.inc => hello.bb} |  0
 meta-isar/recipes-app/hello/hello_2.10.bb           | 11 -----------
 meta-isar/recipes-app/hello/hello_2.9.bb            |  8 --------
 3 files changed, 19 deletions(-)
 rename meta-isar/recipes-app/hello/{hello.inc => hello.bb} (100%)
 delete mode 100644 meta-isar/recipes-app/hello/hello_2.10.bb
 delete mode 100644 meta-isar/recipes-app/hello/hello_2.9.bb

diff --git a/meta-isar/recipes-app/hello/hello.inc b/meta-isar/recipes-app/hello/hello.bb
similarity index 100%
rename from meta-isar/recipes-app/hello/hello.inc
rename to meta-isar/recipes-app/hello/hello.bb
diff --git a/meta-isar/recipes-app/hello/hello_2.10.bb b/meta-isar/recipes-app/hello/hello_2.10.bb
deleted file mode 100644
index bfb8722..0000000
--- a/meta-isar/recipes-app/hello/hello_2.10.bb
+++ /dev/null
@@ -1,11 +0,0 @@
-# Example recipe to rebuild a debian source package
-#
-# This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2019
-#
-# SPDX-License-Identifier: MIT
-
-require hello.inc
-
-DEFAULT_PREFERENCE_debian-buster = "1"
-DEFAULT_PREFERENCE_debian-stretch = "1"
diff --git a/meta-isar/recipes-app/hello/hello_2.9.bb b/meta-isar/recipes-app/hello/hello_2.9.bb
deleted file mode 100644
index 2fe59d1..0000000
--- a/meta-isar/recipes-app/hello/hello_2.9.bb
+++ /dev/null
@@ -1,8 +0,0 @@
-# Example recipe to rebuild a debian source package
-#
-# This software is a part of ISAR.
-# Copyright (c) Siemens AG, 2019
-#
-# SPDX-License-Identifier: MIT
-
-require hello.inc
-- 
2.17.1

Re: [PATCH v3 0/2] Custom ${S} series

[Henning Schild]

LGTM, thanks!

Henning

Am Thu, 4 Mar 2021 23:11:35 +0530
schrieb Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>:

> Changes since V2:
> - Introduce P2 to demonstrate apt:// without ${PV} set.
> - Address review comment from Henning.
> - Fix issue where do_apt_unpack is triggered for non apt://
>   SRC_URIs.
> 
> CI build going on:
> http://ci.isar-build.org:8080/job/isar_vkk_devel/93/
> 
> Changes since V1:
> 
> - Introduce a separate do_apt_unpack task
> 
> Vijai Kumar K (2):
>   dpkg-base: Introduce do_apt_unpack
>   Modify hello to demonstrate apt:// without ${PV}
> 
>  .../recipes-app/hello/{hello.inc => hello.bb} |  0
>  meta-isar/recipes-app/hello/hello_2.10.bb     | 11 -------
>  meta-isar/recipes-app/hello/hello_2.9.bb      |  8 -----
>  meta/classes/dpkg-base.bbclass                | 29
> ++++++++++++++++--- 4 files changed, 25 insertions(+), 23 deletions(-)
>  rename meta-isar/recipes-app/hello/{hello.inc => hello.bb} (100%)
>  delete mode 100644 meta-isar/recipes-app/hello/hello_2.10.bb
>  delete mode 100644 meta-isar/recipes-app/hello/hello_2.9.bb
> 

Re: [PATCH v3 0/2] Custom ${S} series

[Henning Schild]

CI on a downstream layer said that p1 works.

Henning

Am Thu, 4 Mar 2021 19:56:12 +0100
schrieb "[ext] Henning Schild" <henning.schild@siemens.com>:

> LGTM, thanks!
> 
> Henning
> 
> Am Thu, 4 Mar 2021 23:11:35 +0530
> schrieb Vijai Kumar K <Vijaikumar_Kanagarajan@mentor.com>:
> 
> > Changes since V2:
> > - Introduce P2 to demonstrate apt:// without ${PV} set.
> > - Address review comment from Henning.
> > - Fix issue where do_apt_unpack is triggered for non apt://
> >   SRC_URIs.
> > 
> > CI build going on:
> > http://ci.isar-build.org:8080/job/isar_vkk_devel/93/
> > 
> > Changes since V1:
> > 
> > - Introduce a separate do_apt_unpack task
> > 
> > Vijai Kumar K (2):
> >   dpkg-base: Introduce do_apt_unpack
> >   Modify hello to demonstrate apt:// without ${PV}
> > 
> >  .../recipes-app/hello/{hello.inc => hello.bb} |  0
> >  meta-isar/recipes-app/hello/hello_2.10.bb     | 11 -------
> >  meta-isar/recipes-app/hello/hello_2.9.bb      |  8 -----
> >  meta/classes/dpkg-base.bbclass                | 29
> > ++++++++++++++++--- 4 files changed, 25 insertions(+), 23
> > deletions(-) rename meta-isar/recipes-app/hello/{hello.inc =>
> > hello.bb} (100%) delete mode 100644
> > meta-isar/recipes-app/hello/hello_2.10.bb delete mode 100644
> > meta-isar/recipes-app/hello/hello_2.9.bb 
> 

Re: [PATCH v3 0/2] Custom ${S} series

[Anton Mikanovich]

04.03.2021 20:41, Vijai Kumar K wrote:
> Changes since V2:
> - Introduce P2 to demonstrate apt:// without ${PV} set.
> - Address review comment from Henning.
> - Fix issue where do_apt_unpack is triggered for non apt://
>    SRC_URIs.
>
> CI build going on: http://ci.isar-build.org:8080/job/isar_vkk_devel/93/
>
> Changes since V1:
>
> - Introduce a separate do_apt_unpack task
>
> Vijai Kumar K (2):
>    dpkg-base: Introduce do_apt_unpack
>    Modify hello to demonstrate apt:// without ${PV}
>
>   .../recipes-app/hello/{hello.inc => hello.bb} |  0
>   meta-isar/recipes-app/hello/hello_2.10.bb     | 11 -------
>   meta-isar/recipes-app/hello/hello_2.9.bb      |  8 -----
>   meta/classes/dpkg-base.bbclass                | 29 ++++++++++++++++---
>   4 files changed, 25 insertions(+), 23 deletions(-)
>   rename meta-isar/recipes-app/hello/{hello.inc => hello.bb} (100%)
>   delete mode 100644 meta-isar/recipes-app/hello/hello_2.10.bb
>   delete mode 100644 meta-isar/recipes-app/hello/hello_2.9.bb
>
Hello, did you try it with ISAR_CROSS_COMPILE = "0"?
I see it failed on qemumipsel-stretch:

 >ERROR: mc:qemumipsel-stretch:hello-1.0-r0 do_apt_unpack: Execution of 
'build/tmp/work/debian-stretch-mipsel/hello/1.0-r0/temp/run.do_apt_unpack.3153' 
failed with exit code 2:
dpkg-source: error: cannot read hello_2.10-1.dsc: No such file or directory

-- 
Anton Mikanovich
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v3 0/2] Custom ${S} series

[vijaikumar....@gmail.com]

[-- Attachment #1.1: Type: text/plain, Size: 1882 bytes --]


On Monday, March 29, 2021 at 2:44:28 PM UTC+5:30 ami...@ilbers.de wrote:

> 04.03.2021 20:41, Vijai Kumar K wrote: 
> > Changes since V2: 
> > - Introduce P2 to demonstrate apt:// without ${PV} set. 
> > - Address review comment from Henning. 
> > - Fix issue where do_apt_unpack is triggered for non apt:// 
> > SRC_URIs. 
> > 
> > CI build going on: http://ci.isar-build.org:8080/job/isar_vkk_devel/93/ 
> > 
> > Changes since V1: 
> > 
> > - Introduce a separate do_apt_unpack task 
> > 
> > Vijai Kumar K (2): 
> > dpkg-base: Introduce do_apt_unpack 
> > Modify hello to demonstrate apt:// without ${PV} 
> > 
> > .../recipes-app/hello/{hello.inc => hello.bb} | 0 
> > meta-isar/recipes-app/hello/hello_2.10.bb | 11 ------- 
> > meta-isar/recipes-app/hello/hello_2.9.bb | 8 ----- 
> > meta/classes/dpkg-base.bbclass | 29 ++++++++++++++++--- 
> > 4 files changed, 25 insertions(+), 23 deletions(-) 
> > rename meta-isar/recipes-app/hello/{hello.inc => hello.bb} (100%) 
> > delete mode 100644 meta-isar/recipes-app/hello/hello_2.10.bb 
> > delete mode 100644 meta-isar/recipes-app/hello/hello_2.9.bb 
> > 
> Hello, did you try it with ISAR_CROSS_COMPILE = "0"? 
> I see it failed on qemumipsel-stretch: 
>
> >ERROR: mc:qemumipsel-stretch:hello-1.0-r0 do_apt_unpack: Execution of 
> 'build/tmp/work/debian-stretch-mipsel/hello/1.0-r0/temp/run.do_apt_unpack.3153' 
>
> failed with exit code 2: 
> dpkg-source: error: cannot read hello_2.10-1.dsc: No such file or directory


Found the issue. v4 coming up after this[1] pipeline succeeds(hopefully).

[1] http://ci.isar-build.org:8080/job/isar_vkk_devel/95/
 
Thanks,
Vijai Kumar K


>
> -- 
> Anton Mikanovich 
> Promwad Ltd. 
> External service provider of ilbers GmbH 
> Maria-Merian-Str. 8 
> 85521 Ottobrunn, Germany 
> +49 (89) 122 67 24-0 
> Commercial register Munich, HRB 214197 
> General Manager: Baurzhan Ismagulov 
>
>

[-- Attachment #1.2: Type: text/html, Size: 4196 bytes --]