[PATCHv2 0/4] document debianize.bbclass

[PATCHv2 0/4] document debianize.bbclass

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

changes to v1:
 * change example recipe "samefile"
  * more meaningful modifications to the debianization
  * adding more doc comments and updating old ones
  * add DESCRIPTION
 * squash v1 p3+4 into v2 p3

The debianization has long been a hidden feature. Improve it a bit,
document it, add an example and test it in CI.

Henning Schild (4):
  debianize: only prepend changelog when our entry is not there
  debianize: move variable defaults to the class that needs them
  doc: write something about "debianize" and add example package
  CI: add "samefile" to our example config and CI build

 doc/user_manual.md                            | 19 +++++++-
 meta-isar/conf/local.conf.sample              |  2 +-
 .../recipes-app/samefile/samefile_2.14.bb     | 45 +++++++++++++++++++
 meta/classes/debianize.bbclass                | 19 +++++++-
 meta/classes/dpkg-raw.bbclass                 |  3 --
 5 files changed, 82 insertions(+), 6 deletions(-)
 create mode 100644 meta-isar/recipes-app/samefile/samefile_2.14.bb

-- 
2.23.0

[PATCHv2 1/4] debianize: only prepend changelog when our entry is not there

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

The default release and urgency might not fit everyone, make sure we do
not prepend if that very release is already in the changelog. That
allows anyone to choose what they want and prevent prepending.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta/classes/debianize.bbclass | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/meta/classes/debianize.bbclass b/meta/classes/debianize.bbclass
index c231b41..33ab4f7 100644
--- a/meta/classes/debianize.bbclass
+++ b/meta/classes/debianize.bbclass
@@ -21,7 +21,15 @@ ${PN} (${CHANGELOG_V}) UNRELEASED; urgency=low
  -- ${MAINTAINER}  ${date}
 EOF
 	if [ -f ${WORKDIR}/changelog ]; then
-		echo >> ${S}/debian/changelog
+		if head -1 "${WORKDIR}"/changelog | \
+			grep -q -e "^${PN} (${CHANGELOG_V})"
+		then
+			# entry for our version already there, use unmodified
+			rm ${S}/debian/changelog
+		else
+			# prepend our entry to an existing changelog
+			echo >> ${S}/debian/changelog
+		fi
 		cat ${WORKDIR}/changelog >> ${S}/debian/changelog
 	fi
 }
-- 
2.23.0

[PATCHv2 2/4] debianize: move variable defaults to the class that needs them

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta/classes/debianize.bbclass | 3 +++
 meta/classes/dpkg-raw.bbclass  | 3 ---
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/classes/debianize.bbclass b/meta/classes/debianize.bbclass
index 33ab4f7..fb9c234 100644
--- a/meta/classes/debianize.bbclass
+++ b/meta/classes/debianize.bbclass
@@ -5,6 +5,9 @@
 
 CHANGELOG_V ?= "${PV}"
 DPKG_ARCH ??= "any"
+DEBIAN_DEPENDS ?= ""
+DESCRIPTION ?= "must not be empty"
+MAINTAINER ?= "Unknown maintainer <unknown@example.com>"
 
 deb_add_changelog() {
 	timestamp=$(find ${S}/ -type f -not -path "${S}/debian/*" -printf "%T@\n"|sort -n -r|head -n 1)
diff --git a/meta/classes/dpkg-raw.bbclass b/meta/classes/dpkg-raw.bbclass
index 3f19f8e..8c01a46 100644
--- a/meta/classes/dpkg-raw.bbclass
+++ b/meta/classes/dpkg-raw.bbclass
@@ -5,9 +5,6 @@
 
 inherit dpkg
 
-DEBIAN_DEPENDS ?= ""
-MAINTAINER ?= "Unknown maintainer <unknown@example.com>"
-
 D = "${S}"
 
 # Populate folder that will be picked up as package
-- 
2.23.0

[PATCHv2 3/4] doc: write something about "debianize" and add example package

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

Add documentation in the user manual, inside the code and in form of an
example that shows various ways to customize the debianization.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 doc/user_manual.md                            | 19 +++++++-
 .../recipes-app/samefile/samefile_2.14.bb     | 45 +++++++++++++++++++
 meta/classes/debianize.bbclass                |  6 +++
 3 files changed, 69 insertions(+), 1 deletion(-)
 create mode 100644 meta-isar/recipes-app/samefile/samefile_2.14.bb

diff --git a/doc/user_manual.md b/doc/user_manual.md
index e3266a9..e52f5e1 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -590,7 +590,7 @@ Isar currently supports two ways of creating custom packages.
 
 The `deb` packages are built using `dpkg-buildpackage`, so the sources should contain the `debian` directory with necessary meta information. This way is the default way of adding software that needs to be compiled from source. The bbclass for this approach is called `dpkg`.
 
-**NOTE:** If the sources do not contain a `debian` directory your recipe can fetch, create, or ship that.
+**NOTE:** If the sources do not contain a `debian` directory your recipe can fetch, create, or ship that. You might want to read the the next section before returning here.
 
 This is also what you do if you want to rebuild/modify an upstream package.
 Isar does understand `SRC_URI` entries starting with "apt://". For an example
@@ -631,6 +631,23 @@ This approach prevents duplication of the license files in different packages.
 
 The last line in the example above adds recipe to the Isar work chain.
 
+### Compilation of sources missing the debian/-directory
+
+The `debian` directory contains meta information on how to build a package from source. This is roughly speaking "configure", "compile", "install" all described in a Debian-specific way.
+Isar expects your sources to contain the `debian` folder and the above steps need to be described in it, not in a task in a recipe.
+
+So once you have sources you always need to combine them with a `debian` folder before Isar can build a package for you.
+You might be able to find a debianization for a component on the internet, i.e. Ubuntu does package an open source component while Debian does not. Your recipe could download the `debian` folder from Ubuntu and the sources from the open source project.
+
+You can write it yourself, which can be pretty easy but requires a bit of studying. <https://www.debian.org/doc/debian-policy/index.html>
+
+Isar does actually contain a helper that aims to "debianize" sources for your. If your package uses a build-system that Debian knows and follows the well known "configure", "compile", "install" scheme that debianization might just fit your needs without reading Debian manuals.
+If it does not fully fit your needs, it probably gives you a good starting point for your manual tuning.
+
+The shell function `deb_debianize` creates a `debian` folder. But it will not overwrite files that already are in WORKDIR. So you can either just call it to fully generate the `debian` folder. Or you combine it with pre-existing parts.
+
+Have a look at meta-isar/recipes-app/samefile/samefile_2.14.bb and meta/classes/debianize.bbclass for an example and the implementation.
+
 ### Packages without source
 
 If your customization is not about compiling from source there is a second way of creating `deb` packages. That way can be used for cases like:
diff --git a/meta-isar/recipes-app/samefile/samefile_2.14.bb b/meta-isar/recipes-app/samefile/samefile_2.14.bb
new file mode 100644
index 0000000..18a8662
--- /dev/null
+++ b/meta-isar/recipes-app/samefile/samefile_2.14.bb
@@ -0,0 +1,45 @@
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2019
+#
+# SPDX-License-Identifier: MIT
+
+inherit dpkg
+
+DEBIAN_DEPENDS = "\${misc:Depends}"
+DESCRIPTION = "utility that finds files with identical contents"
+
+# These variables allow more control, read the classes to find the default
+# values, or check bitbake -e
+# MAINTAINER CHANGELOG_V DPKG_ARCH
+
+SRC_URI = "http://www.schweikhardt.net/samefile-2.14.tar.gz"
+SRC_URI[md5sum] = "0b438249f3549f18b49cbb49b0473f70"
+
+do_prepare_build[cleandirs] += "${S}/debian"
+do_prepare_build() {
+    # You could also create parts of your debianization before calling
+    # deb_debianize. Pre-exisiting files will not be recreated, changelog
+    # will be prepended unless its latest entry is for CHANGELOG_V.
+    cat << EOF > ${WORKDIR}/changelog
+${PN} (0.1) unstable; urgency=low
+
+  * a long long time ago there was an early version
+
+ -- ${MAINTAINER}  Thu, 24 Dec 1970 00:00:00 +0100
+EOF
+    
+    # Hooks should be placed into WORKDIR before calling deb_debianize.
+    cat << EOF > ${WORKDIR}/postinst
+#!/bin/sh
+echo "" >&2
+echo "NOTE: This package was built by Isar." >&2
+echo "" >&2
+EOF
+
+    # This step creates everything dpkg-buildpackage needs. For further details
+    # you might want to look at its implementation.
+    deb_debianize
+
+    # We can also customize afterwards, in this case change the package section.
+    sed -i -e 's/Section: misc/Section: utils/g' ${S}/debian/control 
+}
diff --git a/meta/classes/debianize.bbclass b/meta/classes/debianize.bbclass
index fb9c234..f07a432 100644
--- a/meta/classes/debianize.bbclass
+++ b/meta/classes/debianize.bbclass
@@ -76,23 +76,29 @@ EOF
 }
 
 deb_debianize() {
+	# create the compat-file if there is no file with that name in WORKDIR
 	if [ -f ${WORKDIR}/compat ]; then
 		install -v -m 644 ${WORKDIR}/compat ${S}/debian/compat
 	else
 		deb_create_compat
 	fi
+	# create the control-file if there is no control-file in WORKDIR
 	if [ -f ${WORKDIR}/control ]; then
 		install -v -m 644 ${WORKDIR}/control ${S}/debian/control
 	else
 		deb_create_control
 	fi
+	# create rules if WORKDIR does not contain a rules-file
 	if [ -f ${WORKDIR}/rules ]; then
 		install -v -m 755 ${WORKDIR}/rules ${S}/debian/rules
 	else
 		deb_create_rules
 	fi
+	# prepend a changelog-entry unless an existing changelog file already
+	# contains an entry with CHANGELOG_V
 	deb_add_changelog
 
+	# copy all hooks from WORKDIR into debian/, hooks are not generated
 	for t in pre post
 	do
 		for a in inst rm
-- 
2.23.0

Re: [PATCHv2 3/4] doc: write something about "debianize" and add example package

[Jan Kiszka]

On 07.11.19 20:13, Henning Schild wrote:
> From: Henning Schild <henning.schild@siemens.com>
> 
> Add documentation in the user manual, inside the code and in form of an
> example that shows various ways to customize the debianization.
> 
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>   doc/user_manual.md                            | 19 +++++++-
>   .../recipes-app/samefile/samefile_2.14.bb     | 45 +++++++++++++++++++
>   meta/classes/debianize.bbclass                |  6 +++
>   3 files changed, 69 insertions(+), 1 deletion(-)
>   create mode 100644 meta-isar/recipes-app/samefile/samefile_2.14.bb
> 
> diff --git a/doc/user_manual.md b/doc/user_manual.md
> index e3266a9..e52f5e1 100644
> --- a/doc/user_manual.md
> +++ b/doc/user_manual.md
> @@ -590,7 +590,7 @@ Isar currently supports two ways of creating custom packages.
>   
>   The `deb` packages are built using `dpkg-buildpackage`, so the sources should contain the `debian` directory with necessary meta information. This way is the default way of adding software that needs to be compiled from source. The bbclass for this approach is called `dpkg`.
>   
> -**NOTE:** If the sources do not contain a `debian` directory your recipe can fetch, create, or ship that.
> +**NOTE:** If the sources do not contain a `debian` directory your recipe can fetch, create, or ship that. You might want to read the the next section before returning here.
>   
>   This is also what you do if you want to rebuild/modify an upstream package.
>   Isar does understand `SRC_URI` entries starting with "apt://". For an example
> @@ -631,6 +631,23 @@ This approach prevents duplication of the license files in different packages.
>   
>   The last line in the example above adds recipe to the Isar work chain.
>   
> +### Compilation of sources missing the debian/-directory
> +
> +The `debian` directory contains meta information on how to build a package from source. This is roughly speaking "configure", "compile", "install" all described in a Debian-specific way.
> +Isar expects your sources to contain the `debian` folder and the above steps need to be described in it, not in a task in a recipe.
> +
> +So once you have sources you always need to combine them with a `debian` folder before Isar can build a package for you.
> +You might be able to find a debianization for a component on the internet, i.e. Ubuntu does package an open source component while Debian does not. Your recipe could download the `debian` folder from Ubuntu and the sources from the open source project.
> +
> +You can write it yourself, which can be pretty easy but requires a bit of studying. <https://www.debian.org/doc/debian-policy/index.html>
> +
> +Isar does actually contain a helper that aims to "debianize" sources for your. If your package uses a build-system that Debian knows and follows the well known "configure", "compile", "install" scheme that debianization might just fit your needs without reading Debian manuals.
> +If it does not fully fit your needs, it probably gives you a good starting point for your manual tuning.
> +
> +The shell function `deb_debianize` creates a `debian` folder. But it will not overwrite files that already are in WORKDIR. So you can either just call it to fully generate the `debian` folder. Or you combine it with pre-existing parts.
> +
> +Have a look at meta-isar/recipes-app/samefile/samefile_2.14.bb and meta/classes/debianize.bbclass for an example and the implementation.
> +
>   ### Packages without source
>   
>   If your customization is not about compiling from source there is a second way of creating `deb` packages. That way can be used for cases like:
> diff --git a/meta-isar/recipes-app/samefile/samefile_2.14.bb b/meta-isar/recipes-app/samefile/samefile_2.14.bb
> new file mode 100644
> index 0000000..18a8662
> --- /dev/null
> +++ b/meta-isar/recipes-app/samefile/samefile_2.14.bb
> @@ -0,0 +1,45 @@
> +# This software is a part of ISAR.
> +# Copyright (c) Siemens AG, 2019
> +#
> +# SPDX-License-Identifier: MIT
> +
> +inherit dpkg
> +
> +DEBIAN_DEPENDS = "\${misc:Depends}"
> +DESCRIPTION = "utility that finds files with identical contents"
> +
> +# These variables allow more control, read the classes to find the default
> +# values, or check bitbake -e
> +# MAINTAINER CHANGELOG_V DPKG_ARCH
> +
> +SRC_URI = "http://www.schweikhardt.net/samefile-2.14.tar.gz"
> +SRC_URI[md5sum] = "0b438249f3549f18b49cbb49b0473f70"
> +
> +do_prepare_build[cleandirs] += "${S}/debian"
> +do_prepare_build() {
> +    # You could also create parts of your debianization before calling
> +    # deb_debianize. Pre-exisiting files will not be recreated, changelog
> +    # will be prepended unless its latest entry is for CHANGELOG_V.
> +    cat << EOF > ${WORKDIR}/changelog
> +${PN} (0.1) unstable; urgency=low
> +
> +  * a long long time ago there was an early version
> +
> + -- ${MAINTAINER}  Thu, 24 Dec 1970 00:00:00 +0100
> +EOF
> +
> +    # Hooks should be placed into WORKDIR before calling deb_debianize.
> +    cat << EOF > ${WORKDIR}/postinst
> +#!/bin/sh
> +echo "" >&2
> +echo "NOTE: This package was built by Isar." >&2
> +echo "" >&2
> +EOF
> +
> +    # This step creates everything dpkg-buildpackage needs. For further details
> +    # you might want to look at its implementation.
> +    deb_debianize
> +
> +    # We can also customize afterwards, in this case change the package section.
> +    sed -i -e 's/Section: misc/Section: utils/g' ${S}/debian/control
> +}
> diff --git a/meta/classes/debianize.bbclass b/meta/classes/debianize.bbclass
> index fb9c234..f07a432 100644
> --- a/meta/classes/debianize.bbclass
> +++ b/meta/classes/debianize.bbclass
> @@ -76,23 +76,29 @@ EOF
>   }
>   
>   deb_debianize() {
> +	# create the compat-file if there is no file with that name in WORKDIR
>   	if [ -f ${WORKDIR}/compat ]; then
>   		install -v -m 644 ${WORKDIR}/compat ${S}/debian/compat
>   	else
>   		deb_create_compat
>   	fi
> +	# create the control-file if there is no control-file in WORKDIR
>   	if [ -f ${WORKDIR}/control ]; then
>   		install -v -m 644 ${WORKDIR}/control ${S}/debian/control
>   	else
>   		deb_create_control
>   	fi
> +	# create rules if WORKDIR does not contain a rules-file
>   	if [ -f ${WORKDIR}/rules ]; then
>   		install -v -m 755 ${WORKDIR}/rules ${S}/debian/rules
>   	else
>   		deb_create_rules
>   	fi
> +	# prepend a changelog-entry unless an existing changelog file already
> +	# contains an entry with CHANGELOG_V
>   	deb_add_changelog
>   
> +	# copy all hooks from WORKDIR into debian/, hooks are not generated
>   	for t in pre post
>   	do
>   		for a in inst rm
> 

Minor: git am gives two trailing whitespace warnings.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

[PATCHv2 4/4] CI: add "samefile" to our example config and CI build

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta-isar/conf/local.conf.sample | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
index 7715ce8..280e007 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -166,7 +166,7 @@ CONF_VERSION = "1"
 
 #
 # The default list of extra packages to be installed.
-IMAGE_INSTALL = "hello-isar example-raw example-module-${KERNEL_NAME} enable-fsck isar-exclude-docs"
+IMAGE_INSTALL = "hello-isar example-raw example-module-${KERNEL_NAME} enable-fsck isar-exclude-docs samefile"
 ## "cache_base_repo" relies on the cache ...
 #IMAGE_INSTALL += "isar-disable-apt-cache"
 ## uses apt:// SRC_URI, which does not work with the cache
-- 
2.23.0

Re: [PATCHv2 0/4] document debianize.bbclass

[Jan Kiszka]

On 07.11.19 20:13, Henning Schild wrote:
> From: Henning Schild <henning.schild@siemens.com>
> 
> changes to v1:
>   * change example recipe "samefile"
>    * more meaningful modifications to the debianization
>    * adding more doc comments and updating old ones
>    * add DESCRIPTION
>   * squash v1 p3+4 into v2 p3
> 
> The debianization has long been a hidden feature. Improve it a bit,
> document it, add an example and test it in CI.
> 
> Henning Schild (4):
>    debianize: only prepend changelog when our entry is not there
>    debianize: move variable defaults to the class that needs them
>    doc: write something about "debianize" and add example package
>    CI: add "samefile" to our example config and CI build
> 
>   doc/user_manual.md                            | 19 +++++++-
>   meta-isar/conf/local.conf.sample              |  2 +-
>   .../recipes-app/samefile/samefile_2.14.bb     | 45 +++++++++++++++++++
>   meta/classes/debianize.bbclass                | 19 +++++++-
>   meta/classes/dpkg-raw.bbclass                 |  3 --
>   5 files changed, 82 insertions(+), 6 deletions(-)
>   create mode 100644 meta-isar/recipes-app/samefile/samefile_2.14.bb
> 

Can this be merged?

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCHv2 0/4] document debianize.bbclass

[Henning Schild]

On Mon, 25 Nov 2019 09:02:36 +0100
Jan Kiszka <jan.kiszka@siemens.com> wrote:

> On 07.11.19 20:13, Henning Schild wrote:
> > From: Henning Schild <henning.schild@siemens.com>
> > 
> > changes to v1:
> >   * change example recipe "samefile"
> >    * more meaningful modifications to the debianization
> >    * adding more doc comments and updating old ones
> >    * add DESCRIPTION
> >   * squash v1 p3+4 into v2 p3
> > 
> > The debianization has long been a hidden feature. Improve it a bit,
> > document it, add an example and test it in CI.
> > 
> > Henning Schild (4):
> >    debianize: only prepend changelog when our entry is not there
> >    debianize: move variable defaults to the class that needs them
> >    doc: write something about "debianize" and add example package
> >    CI: add "samefile" to our example config and CI build
> > 
> >   doc/user_manual.md                            | 19 +++++++-
> >   meta-isar/conf/local.conf.sample              |  2 +-
> >   .../recipes-app/samefile/samefile_2.14.bb     | 45
> > +++++++++++++++++++ meta/classes/debianize.bbclass                |
> > 19 +++++++- meta/classes/dpkg-raw.bbclass                 |  3 --
> >   5 files changed, 82 insertions(+), 6 deletions(-)
> >   create mode 100644 meta-isar/recipes-app/samefile/samefile_2.14.bb
> >   
> 
> Can this be merged?

I guess it got stuck on the whitespace issue in p3, fix on the way.

Henning

> Jan
>