[PATCHv3 0/8] apt-get source support

[PATCHv3 0/8] apt-get source support

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

v2 was sent by Maxim so i gave it a 3

diff to v1:
 - add "apt://" protocol to SRC_URI p7, not squashed
 - add docs
 - change the way i mention Cedric as co-author
 - disable cross-build testing in hello

Henning Schild (8):
  conf: add deb-src entries to all our distro configs
  dpkg-base: introduce an "apt-get source" fetch/unpack step
  meta: move debianization code into a class and into dpkg-base
  debianize: allow changlog version change
  meta-isar/recipes-app: add upstream hello rebuild example
  local.conf: enable rebuilding "hello" for all distros
  meta: change apt source fetcher to hook into SRC_URI
  doc: document "apt://" fetch/unpack feature

 doc/user_manual.md                            |  4 +
 .../conf/distro/debian-stretch-backports.list |  1 +
 meta-isar/conf/distro/raspbian-jessie.list    |  1 +
 meta-isar/conf/local.conf.sample              |  2 +-
 meta-isar/recipes-app/hello/hello.inc         | 23 +++++
 meta-isar/recipes-app/hello/hello_2.10.bb     |  8 ++
 meta-isar/recipes-app/hello/hello_2.9.bb      |  8 ++
 meta/classes/base.bbclass                     | 19 ++++
 meta/classes/debianize.bbclass                | 87 +++++++++++++++++++
 meta/classes/dpkg-base.bbclass                | 20 +++++
 meta/classes/dpkg-raw.bbclass                 | 83 ------------------
 meta/conf/distro/debian-buster.list           |  3 +
 meta/conf/distro/debian-jessie.list           |  3 +
 meta/conf/distro/debian-stretch.list          |  3 +
 14 files changed, 181 insertions(+), 84 deletions(-)
 create mode 100644 meta-isar/recipes-app/hello/hello.inc
 create mode 100644 meta-isar/recipes-app/hello/hello_2.10.bb
 create mode 100644 meta-isar/recipes-app/hello/hello_2.9.bb
 create mode 100644 meta/classes/debianize.bbclass

-- 
2.19.2

[PATCHv3 1/8] conf: add deb-src entries to all our distro configs

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

That will allow us to use "apt-get source" to fetch debian sources from
the mirrors we also get the binary packages from.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta-isar/conf/distro/debian-stretch-backports.list | 1 +
 meta-isar/conf/distro/raspbian-jessie.list          | 1 +
 meta/conf/distro/debian-buster.list                 | 3 +++
 meta/conf/distro/debian-jessie.list                 | 3 +++
 meta/conf/distro/debian-stretch.list                | 3 +++
 5 files changed, 11 insertions(+)

diff --git a/meta-isar/conf/distro/debian-stretch-backports.list b/meta-isar/conf/distro/debian-stretch-backports.list
index 727a835..f606be5 100644
--- a/meta-isar/conf/distro/debian-stretch-backports.list
+++ b/meta-isar/conf/distro/debian-stretch-backports.list
@@ -1 +1,2 @@
 deb	http://ftp.de.debian.org/debian stretch-backports main
+deb-src	http://ftp.de.debian.org/debian stretch-backports main
diff --git a/meta-isar/conf/distro/raspbian-jessie.list b/meta-isar/conf/distro/raspbian-jessie.list
index 792f9c6..0a5af41 100644
--- a/meta-isar/conf/distro/raspbian-jessie.list
+++ b/meta-isar/conf/distro/raspbian-jessie.list
@@ -1 +1,2 @@
 deb	http://archive.raspbian.org/raspbian	jessie	main contrib non-free firmware
+deb-src	http://archive.raspbian.org/raspbian	jessie	main contrib non-free firmware
diff --git a/meta/conf/distro/debian-buster.list b/meta/conf/distro/debian-buster.list
index 18311d8..1e38f1a 100644
--- a/meta/conf/distro/debian-buster.list
+++ b/meta/conf/distro/debian-buster.list
@@ -1,3 +1,6 @@
 deb	http://ftp.de.debian.org/debian buster	main contrib non-free
+deb-src	http://ftp.de.debian.org/debian buster	main contrib non-free
 deb	http://ftp.de.debian.org/debian buster-updates	main contrib non-free
+deb-src	http://ftp.de.debian.org/debian buster-updates	main contrib non-free
 deb	http://security.debian.org	buster/updates	main contrib non-free
+deb-src	http://security.debian.org	buster/updates	main contrib non-free
diff --git a/meta/conf/distro/debian-jessie.list b/meta/conf/distro/debian-jessie.list
index be46a57..5143f4d 100644
--- a/meta/conf/distro/debian-jessie.list
+++ b/meta/conf/distro/debian-jessie.list
@@ -1,3 +1,6 @@
 deb	http://ftp.de.debian.org/debian	jessie	main contrib non-free
+deb-src	http://ftp.de.debian.org/debian	jessie	main contrib non-free
 deb	http://ftp.de.debian.org/debian jessie-updates	main contrib non-free
+deb-src	http://ftp.de.debian.org/debian jessie-updates	main contrib non-free
 deb	http://security.debian.org	jessie/updates	main contrib non-free
+deb-src	http://security.debian.org	jessie/updates	main contrib non-free
diff --git a/meta/conf/distro/debian-stretch.list b/meta/conf/distro/debian-stretch.list
index bfa5cf9..d4f3de0 100644
--- a/meta/conf/distro/debian-stretch.list
+++ b/meta/conf/distro/debian-stretch.list
@@ -1,3 +1,6 @@
 deb	http://ftp.de.debian.org/debian stretch	main contrib non-free
+deb-src	http://ftp.de.debian.org/debian stretch	main contrib non-free
 deb	http://ftp.de.debian.org/debian stretch-updates	main contrib non-free
+deb-src	http://ftp.de.debian.org/debian stretch-updates	main contrib non-free
 deb	http://security.debian.org	stretch/updates	main contrib non-free
+deb-src	http://security.debian.org	stretch/updates	main contrib non-free
-- 
2.19.2

[PATCHv3 2/8] dpkg-base: introduce an "apt-get source" fetch/unpack step

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

This creates a new task to fetch/unpack debian sources from debian
source mirrors. This is done using "apt-get source" and the new variable
APT_SRC to control the arguments.

An example where a original debian package gets rebuild will follow in a
later commit.

based on original patch by Cedric Hombourger <Cedric_Hombourger@mentor.com>

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta/classes/dpkg-base.bbclass | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index f1b127c..2f324f9 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -20,6 +20,25 @@ do_adjust_git[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
 inherit patch
 addtask patch after do_adjust_git before do_build
 
+SRC_APT ?= ""
+
+do_apt_fetch[depends] = "buildchroot-target:do_build"
+
+do_apt_fetch() {
+	if [ -z "${@d.getVar("SRC_APT", True).strip()}" ]; then
+		exit
+	fi
+	dpkg_do_mounts
+	E="${@ bb.utils.export_proxies(d)}"
+	sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} \
+		sh -c 'cd ${PP} && apt-get -y source ${SRC_APT}'
+	dpkg_undo_mounts
+}
+
+addtask apt_fetch after do_unpack before do_patch
+do_apt_fetch[lockfiles] += "${REPO_ISAR_DIR}/isar.lock"
+do_apt_fetch[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
+
 def get_package_srcdir(d):
     s = d.getVar("S", True)
     workdir = d.getVar("WORKDIR", True)
-- 
2.19.2

[PATCHv3 3/8] meta: move debianization code into a class and into dpkg-base

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

This is pure restructuring. Move the code into its own class and include
that in dpkg-base, it was available only in dpkg-raw before.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta/classes/debianize.bbclass | 85 ++++++++++++++++++++++++++++++++++
 meta/classes/dpkg-base.bbclass |  1 +
 meta/classes/dpkg-raw.bbclass  | 83 ---------------------------------
 3 files changed, 86 insertions(+), 83 deletions(-)
 create mode 100644 meta/classes/debianize.bbclass

diff --git a/meta/classes/debianize.bbclass b/meta/classes/debianize.bbclass
new file mode 100644
index 0000000..76b2ed3
--- /dev/null
+++ b/meta/classes/debianize.bbclass
@@ -0,0 +1,85 @@
+# This software is a part of ISAR.
+# Copyright (C) 2017-2019 Siemens AG
+
+deb_add_changelog() {
+	date=$( LANG=C date -R )
+	cat <<EOF > ${S}/debian/changelog
+${PN} (${PV}) UNRELEASED; urgency=low
+
+  * generated by Isar
+
+ -- ${MAINTAINER}  ${date}
+EOF
+	if [ -f ${WORKDIR}/changelog ]; then
+		echo >> ${S}/debian/changelog
+		cat ${WORKDIR}/changelog >> ${S}/debian/changelog
+	fi
+}
+
+deb_create_compat() {
+	echo 9 > ${S}/debian/compat
+}
+
+deb_create_control() {
+	compat=$( cat ${S}/debian/compat )
+	cat << EOF > ${S}/debian/control
+Source: ${PN}
+Section: misc
+Priority: optional
+Standards-Version: 3.9.6
+Maintainer: ${MAINTAINER}
+Build-Depends: debhelper (>= ${compat})
+
+Package: ${PN}
+Architecture: any
+Depends: ${DEBIAN_DEPENDS}
+Description: ${DESCRIPTION}
+EOF
+}
+
+DH_FIXPERM_EXCLUSIONS = \
+    "${@' '.join(['-X ' + x for x in \
+                  (d.getVar('PRESERVE_PERMS', False) or '').split()])}"
+
+deb_create_rules() {
+	cat << EOF > ${S}/debian/rules
+#!/usr/bin/make -f
+
+override_dh_fixperms:
+	dh_fixperms ${DH_FIXPERM_EXCLUSIONS}
+
+%:
+	dh \$@
+EOF
+	chmod +x ${S}/debian/rules
+}
+
+deb_debianize() {
+	if [ -f ${WORKDIR}/compat ]; then
+		install -v -m 644 ${WORKDIR}/compat ${S}/debian/compat
+	else
+		deb_create_compat
+	fi
+	if [ -f ${WORKDIR}/control ]; then
+		install -v -m 644 ${WORKDIR}/control ${S}/debian/control
+	else
+		deb_create_control
+	fi
+	if [ -f ${WORKDIR}/rules ]; then
+		install -v -m 755 ${WORKDIR}/rules ${S}/debian/rules
+	else
+		deb_create_rules
+	fi
+	deb_add_changelog
+
+	for t in pre post
+	do
+		for a in inst rm
+		do
+			if [ -f ${WORKDIR}/${t}${a} ]; then
+				install -v -m 755 ${WORKDIR}/${t}${a} \
+					${S}/debian/${t}${a}
+			fi
+		done
+	done
+}
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index 2f324f9..175dc80 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -4,6 +4,7 @@
 # SPDX-License-Identifier: MIT
 
 inherit buildchroot
+inherit debianize
 
 DEPENDS ?= ""
 
diff --git a/meta/classes/dpkg-raw.bbclass b/meta/classes/dpkg-raw.bbclass
index a9143e9..0434313 100644
--- a/meta/classes/dpkg-raw.bbclass
+++ b/meta/classes/dpkg-raw.bbclass
@@ -17,89 +17,6 @@ do_install[cleandirs] = "${D}"
 do_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
 addtask install after do_unpack before do_prepare_build
 
-deb_add_changelog() {
-	date=$( LANG=C date -R )
-	cat <<EOF > ${D}/debian/changelog
-${PN} (${PV}) UNRELEASED; urgency=low
-
-  * generated by Isar
-
- -- ${MAINTAINER}  ${date}
-EOF
-	if [ -f ${WORKDIR}/changelog ]; then
-		echo >> ${D}/debian/changelog
-		cat ${WORKDIR}/changelog >> ${D}/debian/changelog
-	fi
-}
-
-deb_create_compat() {
-	echo 9 > ${D}/debian/compat
-}
-
-deb_create_control() {
-	compat=$( cat ${D}/debian/compat )
-	cat << EOF > ${D}/debian/control
-Source: ${PN}
-Section: misc
-Priority: optional
-Standards-Version: 3.9.6
-Maintainer: ${MAINTAINER}
-Build-Depends: debhelper (>= ${compat})
-
-Package: ${PN}
-Architecture: any
-Depends: ${DEBIAN_DEPENDS}
-Description: ${DESCRIPTION}
-EOF
-}
-
-DH_FIXPERM_EXCLUSIONS = \
-    "${@' '.join(['-X ' + x for x in \
-                  (d.getVar('PRESERVE_PERMS', False) or '').split()])}"
-
-deb_create_rules() {
-	cat << EOF > ${S}/debian/rules
-#!/usr/bin/make -f
-
-override_dh_fixperms:
-	dh_fixperms ${DH_FIXPERM_EXCLUSIONS}
-
-%:
-	dh \$@
-EOF
-	chmod +x ${S}/debian/rules
-}
-
-deb_debianize() {
-	if [ -f ${WORKDIR}/compat ]; then
-		install -v -m 644 ${WORKDIR}/compat ${D}/debian/compat
-	else
-		deb_create_compat
-	fi
-	if [ -f ${WORKDIR}/control ]; then
-		install -v -m 644 ${WORKDIR}/control ${D}/debian/control
-	else
-		deb_create_control
-	fi
-	if [ -f ${WORKDIR}/rules ]; then
-		install -v -m 755 ${WORKDIR}/rules ${D}/debian/rules
-	else
-		deb_create_rules
-	fi
-	deb_add_changelog
-
-	for t in pre post
-	do
-		for a in inst rm
-		do
-			if [ -f ${WORKDIR}/${t}${a} ]; then
-				install -v -m 755 ${WORKDIR}/${t}${a} \
-					${D}/debian/${t}${a}
-			fi
-		done
-	done
-}
-
 do_prepare_build[cleandirs] += "${D}/debian"
 do_prepare_build() {
 	cd ${D}
-- 
2.19.2

[PATCHv3 4/8] debianize: allow changlog version change

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

Introduce a variable to control the version string we put into a
generated changelog entry.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta/classes/debianize.bbclass | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/meta/classes/debianize.bbclass b/meta/classes/debianize.bbclass
index 76b2ed3..defe3c4 100644
--- a/meta/classes/debianize.bbclass
+++ b/meta/classes/debianize.bbclass
@@ -1,10 +1,12 @@
 # This software is a part of ISAR.
 # Copyright (C) 2017-2019 Siemens AG
 
+CHANGELOG_V ?= "${PV}"
+
 deb_add_changelog() {
 	date=$( LANG=C date -R )
 	cat <<EOF > ${S}/debian/changelog
-${PN} (${PV}) UNRELEASED; urgency=low
+${PN} (${CHANGELOG_V}) UNRELEASED; urgency=low
 
   * generated by Isar
 
-- 
2.19.2

[PATCHv3 5/8] meta-isar/recipes-app: add upstream hello rebuild example

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

This commit introduces a rebuild example of an upstream package. It also
shows how to deal with multiple versions of such a package, depending on
your distro. And in fact we actually have to patch it to rebuild it,
because its build-deps are wrong.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta-isar/recipes-app/hello/hello.inc     | 23 +++++++++++++++++++++++
 meta-isar/recipes-app/hello/hello_2.10.bb |  8 ++++++++
 meta-isar/recipes-app/hello/hello_2.9.bb  |  8 ++++++++
 3 files changed, 39 insertions(+)
 create mode 100644 meta-isar/recipes-app/hello/hello.inc
 create mode 100644 meta-isar/recipes-app/hello/hello_2.10.bb
 create mode 100644 meta-isar/recipes-app/hello/hello_2.9.bb

diff --git a/meta-isar/recipes-app/hello/hello.inc b/meta-isar/recipes-app/hello/hello.inc
new file mode 100644
index 0000000..3976b15
--- /dev/null
+++ b/meta-isar/recipes-app/hello/hello.inc
@@ -0,0 +1,23 @@
+# This software is a part of ISAR.
+
+inherit dpkg
+
+# this will fetch and unpack the sources from upstream debian
+SRC_APT = "${PN}"
+
+MAINTAINER = "Your name here <you@domain.com>"
+CHANGELOG_V = "${PV}-99+isar"
+
+do_prepare_build() {
+	deb_add_changelog
+	# this seems to be a build dep missing in the upstream control file
+	if ! grep texinfo ${S}/debian/control; then
+		sed -i -e 's/Build-Depends:/Build-Depends: texinfo,/g' ${S}/debian/control
+	fi
+}
+
+dpkg_runbuild_prepend() {
+	if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then
+		export DEB_BUILD_OPTIONS="nocheck"
+	fi
+}
diff --git a/meta-isar/recipes-app/hello/hello_2.10.bb b/meta-isar/recipes-app/hello/hello_2.10.bb
new file mode 100644
index 0000000..1598565
--- /dev/null
+++ b/meta-isar/recipes-app/hello/hello_2.10.bb
@@ -0,0 +1,8 @@
+# Example recipe to rebuild a debian source package
+#
+# This software is a part of ISAR.
+
+require hello.inc
+
+DEFAULT_PREFERENCE_debian-buster = "1"
+DEFAULT_PREFERENCE_debian-stretch = "1"
diff --git a/meta-isar/recipes-app/hello/hello_2.9.bb b/meta-isar/recipes-app/hello/hello_2.9.bb
new file mode 100644
index 0000000..b5d5b2e
--- /dev/null
+++ b/meta-isar/recipes-app/hello/hello_2.9.bb
@@ -0,0 +1,8 @@
+# Example recipe to rebuild a debian source package
+#
+# This software is a part of ISAR.
+
+require hello.inc
+
+DEFAULT_PREFERENCE_debian-jessie = "1"
+DEFAULT_PREFERENCE_raspbian-jessie = "1"
-- 
2.19.2

[PATCHv3 6/8] local.conf: enable rebuilding "hello" for all distros

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

This commit includes the rebuild example in our default builds and
therefore into CI.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta-isar/conf/local.conf.sample | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
index a671b20..919bb63 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -161,7 +161,7 @@ CONF_VERSION = "1"
 
 #
 # The default list of extra packages to be installed.
-IMAGE_INSTALL = "hello-isar example-raw example-module-${KERNEL_NAME} enable-fsck"
+IMAGE_INSTALL = "hello-isar example-raw example-module-${KERNEL_NAME} enable-fsck hello"
 
 #
 # Enable cross-compilation support
-- 
2.19.2

[PATCHv3 7/8] meta: change apt source fetcher to hook into SRC_URI

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

This patch allows users to use "apt://" URIs in SRC_URI and cleans up
the interface for recipes.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta-isar/recipes-app/hello/hello.inc |  2 +-
 meta/classes/base.bbclass             | 19 +++++++++++++++++++
 2 files changed, 20 insertions(+), 1 deletion(-)

diff --git a/meta-isar/recipes-app/hello/hello.inc b/meta-isar/recipes-app/hello/hello.inc
index 3976b15..c826ad0 100644
--- a/meta-isar/recipes-app/hello/hello.inc
+++ b/meta-isar/recipes-app/hello/hello.inc
@@ -3,7 +3,7 @@
 inherit dpkg
 
 # this will fetch and unpack the sources from upstream debian
-SRC_APT = "${PN}"
+SRC_URI = "apt://${PN}=${PV}"
 
 MAINTAINER = "Your name here <you@domain.com>"
 CHANGELOG_V = "${PV}-99+isar"
diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index d4082de..6ca93bf 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -96,6 +96,25 @@ python() {
                                   + ws + "root_cleandirs\n")
 }
 
+# filter out all "apt://" URIs out of SRC_URI and stick them into SRC_APT
+python() {
+    src_uri = (d.getVar('SRC_URI', True) or "").split()
+    if len(src_uri) == 0:
+        return
+
+    prefix = "apt://"
+    new_src_uri = []
+    src_apt = []
+    for u in src_uri:
+        if u.startswith(prefix):
+            src_apt.append(u[len(prefix):])
+        else:
+            new_src_uri.append(u)
+
+    d.setVar('SRC_URI', ' '.join(new_src_uri))
+    d.prependVar('SRC_APT', ' '.join(src_apt))
+}
+
 do_fetch[dirs] = "${DL_DIR}"
 do_fetch[file-checksums] = "${@bb.fetch.get_checksum_file_list(d)}"
 do_fetch[vardeps] += "SRCREV"
-- 
2.19.2

[PATCHv3 8/8] doc: document "apt://" fetch/unpack feature

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

Talk about its existance, point to the example, and mention the current
caching limitation.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 doc/user_manual.md | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/doc/user_manual.md b/doc/user_manual.md
index eebcaa9..ebc31c6 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -540,6 +540,9 @@ The `deb` packages are built using `dpkg-buildpackage`, so the sources should co
 
 **NOTE:** If the sources do not contain a `debian` directory your recipe can fetch, create, or ship that.
 
+This is also what you do if you want to rebuild/modify an upstream package.
+Isar does understand `SRC_URI` entries starting with "apt://". For an example
+of a customized upstream package have a look at `meta-isar/recipes-app/hello`.
 
 #### Example
 ```
@@ -753,3 +756,4 @@ bitbake multiconfig:qemuarm-stretch:isar-image-base
 ### Limitation
 
 So far the local base-apt repo is not gpg signed.
+Files fetched with the `SRC_URI` protocol "apt://" are not yet cached.
-- 
2.19.2

Re: [PATCHv3 0/8] apt-get source support

[Henning Schild]

Currently in CI as

http://isar-build.org:8080/job/isar_henning_ilbers-ci/71/

Henning

On Thu, 31 Jan 2019 15:18:08 +0100
Henning Schild <henning.schild@siemens.com> wrote:

> From: Henning Schild <henning.schild@siemens.com>
> 
> v2 was sent by Maxim so i gave it a 3
> 
> diff to v1:
>  - add "apt://" protocol to SRC_URI p7, not squashed
>  - add docs
>  - change the way i mention Cedric as co-author
>  - disable cross-build testing in hello
> 
> Henning Schild (8):
>   conf: add deb-src entries to all our distro configs
>   dpkg-base: introduce an "apt-get source" fetch/unpack step
>   meta: move debianization code into a class and into dpkg-base
>   debianize: allow changlog version change
>   meta-isar/recipes-app: add upstream hello rebuild example
>   local.conf: enable rebuilding "hello" for all distros
>   meta: change apt source fetcher to hook into SRC_URI
>   doc: document "apt://" fetch/unpack feature
> 
>  doc/user_manual.md                            |  4 +
>  .../conf/distro/debian-stretch-backports.list |  1 +
>  meta-isar/conf/distro/raspbian-jessie.list    |  1 +
>  meta-isar/conf/local.conf.sample              |  2 +-
>  meta-isar/recipes-app/hello/hello.inc         | 23 +++++
>  meta-isar/recipes-app/hello/hello_2.10.bb     |  8 ++
>  meta-isar/recipes-app/hello/hello_2.9.bb      |  8 ++
>  meta/classes/base.bbclass                     | 19 ++++
>  meta/classes/debianize.bbclass                | 87
> +++++++++++++++++++ meta/classes/dpkg-base.bbclass                |
> 20 +++++ meta/classes/dpkg-raw.bbclass                 | 83
> ------------------ meta/conf/distro/debian-buster.list           |  3
> + meta/conf/distro/debian-jessie.list           |  3 +
>  meta/conf/distro/debian-stretch.list          |  3 +
>  14 files changed, 181 insertions(+), 84 deletions(-)
>  create mode 100644 meta-isar/recipes-app/hello/hello.inc
>  create mode 100644 meta-isar/recipes-app/hello/hello_2.10.bb
>  create mode 100644 meta-isar/recipes-app/hello/hello_2.9.bb
>  create mode 100644 meta/classes/debianize.bbclass
> 

Re: [PATCHv3 3/8] meta: move debianization code into a class and into dpkg-base

[Jan Kiszka]

On 31.01.19 15:18, [ext] Henning Schild wrote:
> From: Henning Schild <henning.schild@siemens.com>
> 
> This is pure restructuring. Move the code into its own class and include
> that in dpkg-base, it was available only in dpkg-raw before.
> 
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>   meta/classes/debianize.bbclass | 85 ++++++++++++++++++++++++++++++++++
>   meta/classes/dpkg-base.bbclass |  1 +
>   meta/classes/dpkg-raw.bbclass  | 83 ---------------------------------
>   3 files changed, 86 insertions(+), 83 deletions(-)
>   create mode 100644 meta/classes/debianize.bbclass
> 
> diff --git a/meta/classes/debianize.bbclass b/meta/classes/debianize.bbclass
> new file mode 100644
> index 0000000..76b2ed3
> --- /dev/null
> +++ b/meta/classes/debianize.bbclass
> @@ -0,0 +1,85 @@
> +# This software is a part of ISAR.
> +# Copyright (C) 2017-2019 Siemens AG

Missing SPDX-Identifier (i.e. license tag).

> +
> +deb_add_changelog() {
> +	date=$( LANG=C date -R )
> +	cat <<EOF > ${S}/debian/changelog
> +${PN} (${PV}) UNRELEASED; urgency=low
> +
> +  * generated by Isar
> +
> + -- ${MAINTAINER}  ${date}
> +EOF
> +	if [ -f ${WORKDIR}/changelog ]; then
> +		echo >> ${S}/debian/changelog
> +		cat ${WORKDIR}/changelog >> ${S}/debian/changelog
> +	fi
> +}
> +
> +deb_create_compat() {
> +	echo 9 > ${S}/debian/compat
> +}
> +
> +deb_create_control() {
> +	compat=$( cat ${S}/debian/compat )
> +	cat << EOF > ${S}/debian/control
> +Source: ${PN}
> +Section: misc
> +Priority: optional
> +Standards-Version: 3.9.6
> +Maintainer: ${MAINTAINER}
> +Build-Depends: debhelper (>= ${compat})
> +
> +Package: ${PN}
> +Architecture: any
> +Depends: ${DEBIAN_DEPENDS}
> +Description: ${DESCRIPTION}
> +EOF
> +}
> +
> +DH_FIXPERM_EXCLUSIONS = \
> +    "${@' '.join(['-X ' + x for x in \
> +                  (d.getVar('PRESERVE_PERMS', False) or '').split()])}"
> +
> +deb_create_rules() {
> +	cat << EOF > ${S}/debian/rules
> +#!/usr/bin/make -f
> +
> +override_dh_fixperms:
> +	dh_fixperms ${DH_FIXPERM_EXCLUSIONS}
> +
> +%:
> +	dh \$@
> +EOF
> +	chmod +x ${S}/debian/rules
> +}
> +
> +deb_debianize() {
> +	if [ -f ${WORKDIR}/compat ]; then
> +		install -v -m 644 ${WORKDIR}/compat ${S}/debian/compat
> +	else
> +		deb_create_compat
> +	fi
> +	if [ -f ${WORKDIR}/control ]; then
> +		install -v -m 644 ${WORKDIR}/control ${S}/debian/control
> +	else
> +		deb_create_control
> +	fi
> +	if [ -f ${WORKDIR}/rules ]; then
> +		install -v -m 755 ${WORKDIR}/rules ${S}/debian/rules
> +	else
> +		deb_create_rules
> +	fi
> +	deb_add_changelog
> +
> +	for t in pre post
> +	do
> +		for a in inst rm
> +		do
> +			if [ -f ${WORKDIR}/${t}${a} ]; then
> +				install -v -m 755 ${WORKDIR}/${t}${a} \
> +					${S}/debian/${t}${a}
> +			fi
> +		done
> +	done
> +}
> diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
> index 2f324f9..175dc80 100644
> --- a/meta/classes/dpkg-base.bbclass
> +++ b/meta/classes/dpkg-base.bbclass
> @@ -4,6 +4,7 @@
>   # SPDX-License-Identifier: MIT
>   
>   inherit buildchroot
> +inherit debianize
>   
>   DEPENDS ?= ""
>   
> diff --git a/meta/classes/dpkg-raw.bbclass b/meta/classes/dpkg-raw.bbclass
> index a9143e9..0434313 100644
> --- a/meta/classes/dpkg-raw.bbclass
> +++ b/meta/classes/dpkg-raw.bbclass
> @@ -17,89 +17,6 @@ do_install[cleandirs] = "${D}"
>   do_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>   addtask install after do_unpack before do_prepare_build
>   
> -deb_add_changelog() {
> -	date=$( LANG=C date -R )
> -	cat <<EOF > ${D}/debian/changelog
> -${PN} (${PV}) UNRELEASED; urgency=low
> -
> -  * generated by Isar
> -
> - -- ${MAINTAINER}  ${date}
> -EOF
> -	if [ -f ${WORKDIR}/changelog ]; then
> -		echo >> ${D}/debian/changelog
> -		cat ${WORKDIR}/changelog >> ${D}/debian/changelog
> -	fi
> -}
> -
> -deb_create_compat() {
> -	echo 9 > ${D}/debian/compat
> -}
> -
> -deb_create_control() {
> -	compat=$( cat ${D}/debian/compat )
> -	cat << EOF > ${D}/debian/control
> -Source: ${PN}
> -Section: misc
> -Priority: optional
> -Standards-Version: 3.9.6
> -Maintainer: ${MAINTAINER}
> -Build-Depends: debhelper (>= ${compat})
> -
> -Package: ${PN}
> -Architecture: any
> -Depends: ${DEBIAN_DEPENDS}
> -Description: ${DESCRIPTION}
> -EOF
> -}
> -
> -DH_FIXPERM_EXCLUSIONS = \
> -    "${@' '.join(['-X ' + x for x in \
> -                  (d.getVar('PRESERVE_PERMS', False) or '').split()])}"
> -
> -deb_create_rules() {
> -	cat << EOF > ${S}/debian/rules
> -#!/usr/bin/make -f
> -
> -override_dh_fixperms:
> -	dh_fixperms ${DH_FIXPERM_EXCLUSIONS}
> -
> -%:
> -	dh \$@
> -EOF
> -	chmod +x ${S}/debian/rules
> -}
> -
> -deb_debianize() {
> -	if [ -f ${WORKDIR}/compat ]; then
> -		install -v -m 644 ${WORKDIR}/compat ${D}/debian/compat
> -	else
> -		deb_create_compat
> -	fi
> -	if [ -f ${WORKDIR}/control ]; then
> -		install -v -m 644 ${WORKDIR}/control ${D}/debian/control
> -	else
> -		deb_create_control
> -	fi
> -	if [ -f ${WORKDIR}/rules ]; then
> -		install -v -m 755 ${WORKDIR}/rules ${D}/debian/rules
> -	else
> -		deb_create_rules
> -	fi
> -	deb_add_changelog
> -
> -	for t in pre post
> -	do
> -		for a in inst rm
> -		do
> -			if [ -f ${WORKDIR}/${t}${a} ]; then
> -				install -v -m 755 ${WORKDIR}/${t}${a} \
> -					${D}/debian/${t}${a}
> -			fi
> -		done
> -	done
> -}
> -
>   do_prepare_build[cleandirs] += "${D}/debian"
>   do_prepare_build() {
>   	cd ${D}
> 

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCHv3 5/8] meta-isar/recipes-app: add upstream hello rebuild example

[Jan Kiszka]

On 31.01.19 15:18, [ext] Henning Schild wrote:
> From: Henning Schild <henning.schild@siemens.com>
> 
> This commit introduces a rebuild example of an upstream package. It also
> shows how to deal with multiple versions of such a package, depending on
> your distro. And in fact we actually have to patch it to rebuild it,
> because its build-deps are wrong.
> 
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>   meta-isar/recipes-app/hello/hello.inc     | 23 +++++++++++++++++++++++
>   meta-isar/recipes-app/hello/hello_2.10.bb |  8 ++++++++
>   meta-isar/recipes-app/hello/hello_2.9.bb  |  8 ++++++++
>   3 files changed, 39 insertions(+)
>   create mode 100644 meta-isar/recipes-app/hello/hello.inc
>   create mode 100644 meta-isar/recipes-app/hello/hello_2.10.bb
>   create mode 100644 meta-isar/recipes-app/hello/hello_2.9.bb
> 
> diff --git a/meta-isar/recipes-app/hello/hello.inc b/meta-isar/recipes-app/hello/hello.inc
> new file mode 100644
> index 0000000..3976b15
> --- /dev/null
> +++ b/meta-isar/recipes-app/hello/hello.inc
> @@ -0,0 +1,23 @@
> +# This software is a part of ISAR.
> +
> +inherit dpkg
> +
> +# this will fetch and unpack the sources from upstream debian
> +SRC_APT = "${PN}"
> +
> +MAINTAINER = "Your name here <you@domain.com>"
> +CHANGELOG_V = "${PV}-99+isar"
> +
> +do_prepare_build() {
> +	deb_add_changelog
> +	# this seems to be a build dep missing in the upstream control file
> +	if ! grep texinfo ${S}/debian/control; then
> +		sed -i -e 's/Build-Depends:/Build-Depends: texinfo,/g' ${S}/debian/control
> +	fi
> +}
> +
> +dpkg_runbuild_prepend() {
> +	if [ ${ISAR_CROSS_COMPILE} -eq 1 ]; then
> +		export DEB_BUILD_OPTIONS="nocheck"
> +	fi
> +}
> diff --git a/meta-isar/recipes-app/hello/hello_2.10.bb b/meta-isar/recipes-app/hello/hello_2.10.bb
> new file mode 100644
> index 0000000..1598565
> --- /dev/null
> +++ b/meta-isar/recipes-app/hello/hello_2.10.bb
> @@ -0,0 +1,8 @@
> +# Example recipe to rebuild a debian source package
> +#
> +# This software is a part of ISAR.
> +
> +require hello.inc
> +
> +DEFAULT_PREFERENCE_debian-buster = "1"
> +DEFAULT_PREFERENCE_debian-stretch = "1"
> diff --git a/meta-isar/recipes-app/hello/hello_2.9.bb b/meta-isar/recipes-app/hello/hello_2.9.bb
> new file mode 100644
> index 0000000..b5d5b2e
> --- /dev/null
> +++ b/meta-isar/recipes-app/hello/hello_2.9.bb
> @@ -0,0 +1,8 @@
> +# Example recipe to rebuild a debian source package
> +#
> +# This software is a part of ISAR.
> +
> +require hello.inc
> +
> +DEFAULT_PREFERENCE_debian-jessie = "1"
> +DEFAULT_PREFERENCE_raspbian-jessie = "1"
> 

Incomplete file headers: no copyright, no license tags.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCHv3 0/8] apt-get source support

[Henning Schild]

On Thu, 31 Jan 2019 15:20:54 +0100
"[ext] Henning Schild" <henning.schild@siemens.com> wrote:

> Currently in CI as
> 
> http://isar-build.org:8080/job/isar_henning_ilbers-ci/71/

Failed because of known offline build issue. New series will skip
building the package in offline mode.

Henning

> Henning
> 
> On Thu, 31 Jan 2019 15:18:08 +0100
> Henning Schild <henning.schild@siemens.com> wrote:
> 
> > From: Henning Schild <henning.schild@siemens.com>
> > 
> > v2 was sent by Maxim so i gave it a 3
> > 
> > diff to v1:
> >  - add "apt://" protocol to SRC_URI p7, not squashed
> >  - add docs
> >  - change the way i mention Cedric as co-author
> >  - disable cross-build testing in hello
> > 
> > Henning Schild (8):
> >   conf: add deb-src entries to all our distro configs
> >   dpkg-base: introduce an "apt-get source" fetch/unpack step
> >   meta: move debianization code into a class and into dpkg-base
> >   debianize: allow changlog version change
> >   meta-isar/recipes-app: add upstream hello rebuild example
> >   local.conf: enable rebuilding "hello" for all distros
> >   meta: change apt source fetcher to hook into SRC_URI
> >   doc: document "apt://" fetch/unpack feature
> > 
> >  doc/user_manual.md                            |  4 +
> >  .../conf/distro/debian-stretch-backports.list |  1 +
> >  meta-isar/conf/distro/raspbian-jessie.list    |  1 +
> >  meta-isar/conf/local.conf.sample              |  2 +-
> >  meta-isar/recipes-app/hello/hello.inc         | 23 +++++
> >  meta-isar/recipes-app/hello/hello_2.10.bb     |  8 ++
> >  meta-isar/recipes-app/hello/hello_2.9.bb      |  8 ++
> >  meta/classes/base.bbclass                     | 19 ++++
> >  meta/classes/debianize.bbclass                | 87
> > +++++++++++++++++++ meta/classes/dpkg-base.bbclass                |
> > 20 +++++ meta/classes/dpkg-raw.bbclass                 | 83
> > ------------------ meta/conf/distro/debian-buster.list           |
> > 3
> > + meta/conf/distro/debian-jessie.list           |  3 +
> >  meta/conf/distro/debian-stretch.list          |  3 +
> >  14 files changed, 181 insertions(+), 84 deletions(-)
> >  create mode 100644 meta-isar/recipes-app/hello/hello.inc
> >  create mode 100644 meta-isar/recipes-app/hello/hello_2.10.bb
> >  create mode 100644 meta-isar/recipes-app/hello/hello_2.9.bb
> >  create mode 100644 meta/classes/debianize.bbclass
> >   
>