[PATCH 0/5] document debianize.bbclass

[PATCH 0/5] document debianize.bbclass

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

The debianization has long been a hidden feature. Improve it a bit,
document it, add an example and test it in CI.

Henning Schild (5):
  debianize: only prepend changelog when our entry is not there
  debianize: move variable defaults to the class that needs them
  meta-isar/recipes-app: add debianize test package "samefile"
  doc: write something about "debianize"
  CI: add "samefile" to our example config and CI build

 doc/user_manual.md                            | 19 +++++++++++-
 meta-isar/conf/local.conf.sample              |  2 +-
 .../recipes-app/samefile/samefile_2.14.bb     | 30 +++++++++++++++++++
 meta/classes/debianize.bbclass                | 19 +++++++++++-
 meta/classes/dpkg-raw.bbclass                 |  3 --
 5 files changed, 67 insertions(+), 6 deletions(-)
 create mode 100644 meta-isar/recipes-app/samefile/samefile_2.14.bb

-- 
2.23.0

[PATCH 1/5] debianize: only prepend changelog when our entry is not there

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

The default release and urgency might not fit everyone, make sure we do
not prepend if that very release is already in the changelog. That
allows anyone to choose what they want and prevent prepending.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta/classes/debianize.bbclass | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/meta/classes/debianize.bbclass b/meta/classes/debianize.bbclass
index c231b41..33ab4f7 100644
--- a/meta/classes/debianize.bbclass
+++ b/meta/classes/debianize.bbclass
@@ -21,7 +21,15 @@ ${PN} (${CHANGELOG_V}) UNRELEASED; urgency=low
  -- ${MAINTAINER}  ${date}
 EOF
 	if [ -f ${WORKDIR}/changelog ]; then
-		echo >> ${S}/debian/changelog
+		if head -1 "${WORKDIR}"/changelog | \
+			grep -q -e "^${PN} (${CHANGELOG_V})"
+		then
+			# entry for our version already there, use unmodified
+			rm ${S}/debian/changelog
+		else
+			# prepend our entry to an existing changelog
+			echo >> ${S}/debian/changelog
+		fi
 		cat ${WORKDIR}/changelog >> ${S}/debian/changelog
 	fi
 }
-- 
2.23.0

[PATCH 2/5] debianize: move variable defaults to the class that needs them

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta/classes/debianize.bbclass | 3 +++
 meta/classes/dpkg-raw.bbclass  | 3 ---
 2 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/classes/debianize.bbclass b/meta/classes/debianize.bbclass
index 33ab4f7..fb9c234 100644
--- a/meta/classes/debianize.bbclass
+++ b/meta/classes/debianize.bbclass
@@ -5,6 +5,9 @@
 
 CHANGELOG_V ?= "${PV}"
 DPKG_ARCH ??= "any"
+DEBIAN_DEPENDS ?= ""
+DESCRIPTION ?= "must not be empty"
+MAINTAINER ?= "Unknown maintainer <unknown@example.com>"
 
 deb_add_changelog() {
 	timestamp=$(find ${S}/ -type f -not -path "${S}/debian/*" -printf "%T@\n"|sort -n -r|head -n 1)
diff --git a/meta/classes/dpkg-raw.bbclass b/meta/classes/dpkg-raw.bbclass
index 3f19f8e..8c01a46 100644
--- a/meta/classes/dpkg-raw.bbclass
+++ b/meta/classes/dpkg-raw.bbclass
@@ -5,9 +5,6 @@
 
 inherit dpkg
 
-DEBIAN_DEPENDS ?= ""
-MAINTAINER ?= "Unknown maintainer <unknown@example.com>"
-
 D = "${S}"
 
 # Populate folder that will be picked up as package
-- 
2.23.0

Re: [PATCH 2/5] debianize: move variable defaults to the class that needs them

[Jan Kiszka]

On 29.10.19 17:44, Henning Schild wrote:
> From: Henning Schild <henning.schild@siemens.com>
> 
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>  meta/classes/debianize.bbclass | 3 +++
>  meta/classes/dpkg-raw.bbclass  | 3 ---
>  2 files changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/meta/classes/debianize.bbclass b/meta/classes/debianize.bbclass
> index 33ab4f7..fb9c234 100644
> --- a/meta/classes/debianize.bbclass
> +++ b/meta/classes/debianize.bbclass
> @@ -5,6 +5,9 @@
>  
>  CHANGELOG_V ?= "${PV}"
>  DPKG_ARCH ??= "any"
> +DEBIAN_DEPENDS ?= ""
> +DESCRIPTION ?= "must not be empty"

Then, shouldn't we rather leave it empty here and fail if it remains so?

> +MAINTAINER ?= "Unknown maintainer <unknown@example.com>"
>  
>  deb_add_changelog() {
>  	timestamp=$(find ${S}/ -type f -not -path "${S}/debian/*" -printf "%T@\n"|sort -n -r|head -n 1)
> diff --git a/meta/classes/dpkg-raw.bbclass b/meta/classes/dpkg-raw.bbclass
> index 3f19f8e..8c01a46 100644
> --- a/meta/classes/dpkg-raw.bbclass
> +++ b/meta/classes/dpkg-raw.bbclass
> @@ -5,9 +5,6 @@
>  
>  inherit dpkg
>  
> -DEBIAN_DEPENDS ?= ""
> -MAINTAINER ?= "Unknown maintainer <unknown@example.com>"
> -
>  D = "${S}"
>  
>  # Populate folder that will be picked up as package
> 

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH 2/5] debianize: move variable defaults to the class that needs them

[Henning Schild]

Am Wed, 6 Nov 2019 17:20:10 +0100
schrieb Jan Kiszka <jan.kiszka@siemens.com>:

> On 29.10.19 17:44, Henning Schild wrote:
> > From: Henning Schild <henning.schild@siemens.com>
> > 
> > Signed-off-by: Henning Schild <henning.schild@siemens.com>
> > ---
> >  meta/classes/debianize.bbclass | 3 +++
> >  meta/classes/dpkg-raw.bbclass  | 3 ---
> >  2 files changed, 3 insertions(+), 3 deletions(-)
> > 
> > diff --git a/meta/classes/debianize.bbclass
> > b/meta/classes/debianize.bbclass index 33ab4f7..fb9c234 100644
> > --- a/meta/classes/debianize.bbclass
> > +++ b/meta/classes/debianize.bbclass
> > @@ -5,6 +5,9 @@
> >  
> >  CHANGELOG_V ?= "${PV}"
> >  DPKG_ARCH ??= "any"
> > +DEBIAN_DEPENDS ?= ""
> > +DESCRIPTION ?= "must not be empty"  
> 
> Then, shouldn't we rather leave it empty here and fail if it remains
> so?

Not sure, and guess not. We have working but incorrect defaults for all
values, see the silly MAINTAINER below. Will keep it like that for now.

Henning

> > +MAINTAINER ?= "Unknown maintainer <unknown@example.com>"
> >  
> >  deb_add_changelog() {
> >  	timestamp=$(find ${S}/ -type f -not -path "${S}/debian/*"
> > -printf "%T@\n"|sort -n -r|head -n 1) diff --git
> > a/meta/classes/dpkg-raw.bbclass b/meta/classes/dpkg-raw.bbclass
> > index 3f19f8e..8c01a46 100644 --- a/meta/classes/dpkg-raw.bbclass
> > +++ b/meta/classes/dpkg-raw.bbclass
> > @@ -5,9 +5,6 @@
> >  
> >  inherit dpkg
> >  
> > -DEBIAN_DEPENDS ?= ""
> > -MAINTAINER ?= "Unknown maintainer <unknown@example.com>"
> > -
> >  D = "${S}"
> >  
> >  # Populate folder that will be picked up as package
> >   
> 
> Jan
> 

[PATCH 3/5] meta-isar/recipes-app: add debianize test package "samefile"

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

Thas is a small utility which is fit to demonstrate the usage of the
debianization class.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta-isar/recipes-app/samefile/samefile_2.14.bb | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
 create mode 100644 meta-isar/recipes-app/samefile/samefile_2.14.bb

diff --git a/meta-isar/recipes-app/samefile/samefile_2.14.bb b/meta-isar/recipes-app/samefile/samefile_2.14.bb
new file mode 100644
index 0000000..a24d574
--- /dev/null
+++ b/meta-isar/recipes-app/samefile/samefile_2.14.bb
@@ -0,0 +1,16 @@
+# This software is a part of ISAR.
+# Copyright (c) Siemens AG, 2019
+#
+# SPDX-License-Identifier: MIT
+
+inherit dpkg
+
+DEBIAN_DEPENDS = "\${misc:Depends}"
+
+SRC_URI = "http://www.schweikhardt.net/samefile-2.14.tar.gz"
+SRC_URI[md5sum] = "0b438249f3549f18b49cbb49b0473f70"
+
+do_prepare_build[cleandirs] += "${S}/debian"
+do_prepare_build() {
+    deb_debianize
+}
-- 
2.23.0

[PATCH 4/5] doc: write something about "debianize"

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

Also add some comments to the implementation and extend the example
recipe with some pre- and post-processing.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 doc/user_manual.md                            | 19 ++++++++++++++++++-
 .../recipes-app/samefile/samefile_2.14.bb     | 14 ++++++++++++++
 meta/classes/debianize.bbclass                |  6 ++++++
 3 files changed, 38 insertions(+), 1 deletion(-)

diff --git a/doc/user_manual.md b/doc/user_manual.md
index 34f50df..7c23fcc 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -583,7 +583,7 @@ Isar currently supports two ways of creating custom packages.
 
 The `deb` packages are built using `dpkg-buildpackage`, so the sources should contain the `debian` directory with necessary meta information. This way is the default way of adding software that needs to be compiled from source. The bbclass for this approach is called `dpkg`.
 
-**NOTE:** If the sources do not contain a `debian` directory your recipe can fetch, create, or ship that.
+**NOTE:** If the sources do not contain a `debian` directory your recipe can fetch, create, or ship that. You might want to read the the next section before returning here.
 
 This is also what you do if you want to rebuild/modify an upstream package.
 Isar does understand `SRC_URI` entries starting with "apt://". For an example
@@ -624,6 +624,23 @@ This approach prevents duplication of the license files in different packages.
 
 The last line in the example above adds recipe to the Isar work chain.
 
+### Compilation of sources missing the debian/-directory
+
+The `debian` directory contains meta information on how to build a package from source. This is roughly speaking "configure", "compile", "install" all described in a Debian-specific way.
+Isar expects your sources to contain the `debian` folder and the above steps need to be described in it, not in a task in a recipe.
+
+So once you have sources you always need to combine them with a `debian` folder before Isar can build a package for you.
+You might be able to find a debianization for a component on the internet, i.e. Ubuntu does package an open source component while Debian does not. Your recipe could download the `debian` folder from Ubuntu and the sources from the open source project.
+
+You can write it yourself, which can be pretty easy but requires a bit of studying. <https://www.debian.org/doc/debian-policy/index.html>
+
+Isar does actually contain a helper that aims to "debianize" sources for your. If your package uses a build-system that Debian knows and follows the well known "configure", "compile", "install" scheme that debianization might just fit your needs without reading Debian manuals.
+If it does not fully fit your needs, it probably gives you a good starting point for your manual tuning.
+
+The shell function `deb_debianize` creates a `debian` folder. But it will not overwrite files that already are in WORKDIR. So you can either just call it to fully generate the `debian` folder. Or you combine it with pre-existing parts.
+
+Have a look at meta-isar/recipes-app/samefile/samefile_2.14.bb and meta/classes/debianize.bbclass for an example and the implementation.
+
 ### Packages without source
 
 If your customization is not about compiling from source there is a second way of creating `deb` packages. That way can be used for cases like:
diff --git a/meta-isar/recipes-app/samefile/samefile_2.14.bb b/meta-isar/recipes-app/samefile/samefile_2.14.bb
index a24d574..0ace85a 100644
--- a/meta-isar/recipes-app/samefile/samefile_2.14.bb
+++ b/meta-isar/recipes-app/samefile/samefile_2.14.bb
@@ -12,5 +12,19 @@ SRC_URI[md5sum] = "0b438249f3549f18b49cbb49b0473f70"
 
 do_prepare_build[cleandirs] += "${S}/debian"
 do_prepare_build() {
+    # you could also create parts of your debianization before
+    # pre-exisiting files will not be recreated
+    # 42 is not actually valid ... this is just a stupid example
+    echo 42 > "${WORKDIR}"/compat
+
+    # this step creates everything dpkg-buildpackage needs, compat will not be
+    # overwritten
     deb_debianize
+
+    # now if you are not happy you can post-process the result
+    # now let us "repair" the compat
+    echo 9 > "${S}/debian/compat"
+    # compat happens to be used by this step, so do that again
+    # see debianize.bbclass
+    deb_create_control
 }
diff --git a/meta/classes/debianize.bbclass b/meta/classes/debianize.bbclass
index fb9c234..e1c79ab 100644
--- a/meta/classes/debianize.bbclass
+++ b/meta/classes/debianize.bbclass
@@ -76,23 +76,29 @@ EOF
 }
 
 deb_debianize() {
+	# create the compat-file if there is no file with that name in WORKDIR
 	if [ -f ${WORKDIR}/compat ]; then
 		install -v -m 644 ${WORKDIR}/compat ${S}/debian/compat
 	else
 		deb_create_compat
 	fi
+	# create the control-file if there is no contro-file in WORKDIR
 	if [ -f ${WORKDIR}/control ]; then
 		install -v -m 644 ${WORKDIR}/control ${S}/debian/control
 	else
 		deb_create_control
 	fi
+	# create rules if WORKDIR does not contain a rules-file
 	if [ -f ${WORKDIR}/rules ]; then
 		install -v -m 755 ${WORKDIR}/rules ${S}/debian/rules
 	else
 		deb_create_rules
 	fi
+	# prepend a changelog-entry unless an existing changelog file already
+	# contains an entry with CHANGELOG_V
 	deb_add_changelog
 
+	# copy all hooks from WORKDIR into debian/, hooks are not generated
 	for t in pre post
 	do
 		for a in inst rm
-- 
2.23.0

Re: [PATCH 4/5] doc: write something about "debianize"

[chombourger]

[-- Attachment #1.1: Type: text/plain, Size: 6440 bytes --]



On Tuesday, October 29, 2019 at 5:44:28 PM UTC+1, Henning Schild wrote:
>
> From: Henning Schild <hennin...@siemens.com <javascript:>> 
>
> Also add some comments to the implementation and extend the example 
> recipe with some pre- and post-processing. 
>
> Signed-off-by: Henning Schild <hennin...@siemens.com <javascript:>> 
> --- 
>  doc/user_manual.md                            | 19 ++++++++++++++++++- 
>  .../recipes-app/samefile/samefile_2.14.bb     | 14 ++++++++++++++ 
>  meta/classes/debianize.bbclass                |  6 ++++++ 
>  3 files changed, 38 insertions(+), 1 deletion(-) 
>
> diff --git a/doc/user_manual.md b/doc/user_manual.md 
> index 34f50df..7c23fcc 100644 
> --- a/doc/user_manual.md 
> +++ b/doc/user_manual.md 
> @@ -583,7 +583,7 @@ Isar currently supports two ways of creating custom 
> packages. 
>   
>  The `deb` packages are built using `dpkg-buildpackage`, so the sources 
> should contain the `debian` directory with necessary meta information. This 
> way is the default way of adding software that needs to be compiled from 
> source. The bbclass for this approach is called `dpkg`. 
>   
> -**NOTE:** If the sources do not contain a `debian` directory your recipe 
> can fetch, create, or ship that. 
> +**NOTE:** If the sources do not contain a `debian` directory your recipe 
> can fetch, create, or ship that. You might want to read the the next 
> section before returning here. 
>   
>  This is also what you do if you want to rebuild/modify an upstream 
> package. 
>  Isar does understand `SRC_URI` entries starting with "apt://". For an 
> example 
> @@ -624,6 +624,23 @@ This approach prevents duplication of the license 
> files in different packages. 
>   
>  The last line in the example above adds recipe to the Isar work chain. 
>   
> +### Compilation of sources missing the debian/-directory 
> + 
> +The `debian` directory contains meta information on how to build a 
> package from source. This is roughly speaking "configure", "compile", 
> "install" all described in a Debian-specific way. 
> +Isar expects your sources to contain the `debian` folder and the above 
> steps need to be described in it, not in a task in a recipe. 
> + 
> +So once you have sources you always need to combine them with a `debian` 
> folder before Isar can build a package for you. 
> +You might be able to find a debianization for a component on the 
> internet, i.e. Ubuntu does package an open source component while Debian 
> does not. Your recipe could download the `debian` folder from Ubuntu and 
> the sources from the open source project. 
> + 
> +You can write it yourself, which can be pretty easy but requires a bit of 
> studying. <https://www.debian.org/doc/debian-policy/index.html> 
> + 
> +Isar does actually contain a helper that aims to "debianize" sources for 
> your. If your package uses a build-system that Debian knows and follows the 
> well known "configure", "compile", "install" scheme that debianization 
> might just fit your needs without reading Debian manuals. 
> +If it does not fully fit your needs, it probably gives you a good 
> starting point for your manual tuning. 
> + 
> +The shell function `deb_debianize` creates a `debian` folder. But it will 
> not overwrite files that already are in WORKDIR. So you can either just 
> call it to fully generate the `debian` folder. Or you combine it with 
> pre-existing parts. 
> + 
> +Have a look at meta-isar/recipes-app/samefile/samefile_2.14.bb and 
> meta/classes/debianize.bbclass for an example and the implementation. 
> + 
>  ### Packages without source 
>   
>  If your customization is not about compiling from source there is a 
> second way of creating `deb` packages. That way can be used for cases like: 
> diff --git a/meta-isar/recipes-app/samefile/samefile_2.14.bb 
> b/meta-isar/recipes-app/samefile/samefile_2.14.bb 
> index a24d574..0ace85a 100644 
> --- a/meta-isar/recipes-app/samefile/samefile_2.14.bb 
> +++ b/meta-isar/recipes-app/samefile/samefile_2.14.bb 
> @@ -12,5 +12,19 @@ SRC_URI[md5sum] = "0b438249f3549f18b49cbb49b0473f70" 
>   
>  do_prepare_build[cleandirs] += "${S}/debian" 
>  do_prepare_build() { 
> +    # you could also create parts of your debianization before 
> +    # pre-exisiting files will not be recreated 
> +    # 42 is not actually valid ... this is just a stupid example 
> +    echo 42 > "${WORKDIR}"/compat 
> + 
> +    # this step creates everything dpkg-buildpackage needs, compat will 
> not be 
> +    # overwritten 
>      deb_debianize 
> + 
>

do we want to "read compat <${S}/compat" before we overwrite it just to 
make sure it is still 42? and bbfatal if it isn't?
in other words is this recipe more of a test or of an example?
(either way is fine by me)
 

> +    # now if you are not happy you can post-process the result 
> +    # now let us "repair" the compat 
> +    echo 9 > "${S}/debian/compat" 
> +    # compat happens to be used by this step, so do that again 
> +    # see debianize.bbclass 
> +    deb_create_control 
>  } 
> diff --git a/meta/classes/debianize.bbclass 
> b/meta/classes/debianize.bbclass 
> index fb9c234..e1c79ab 100644 
> --- a/meta/classes/debianize.bbclass 
> +++ b/meta/classes/debianize.bbclass 
> @@ -76,23 +76,29 @@ EOF 
>  } 
>   
>  deb_debianize() { 
> +        # create the compat-file if there is no file with that name in 
> WORKDIR 
>          if [ -f ${WORKDIR}/compat ]; then 
>                  install -v -m 644 ${WORKDIR}/compat ${S}/debian/compat 
>          else 
>                  deb_create_compat 
>          fi 
> +        # create the control-file if there is no contro-file in WORKDIR 
>          if [ -f ${WORKDIR}/control ]; then 
>                  install -v -m 644 ${WORKDIR}/control ${S}/debian/control 
>          else 
>                  deb_create_control 
>          fi 
> +        # create rules if WORKDIR does not contain a rules-file 
>          if [ -f ${WORKDIR}/rules ]; then 
>                  install -v -m 755 ${WORKDIR}/rules ${S}/debian/rules 
>          else 
>                  deb_create_rules 
>          fi 
> +        # prepend a changelog-entry unless an existing changelog file 
> already 
> +        # contains an entry with CHANGELOG_V 
>          deb_add_changelog 
>   
> +        # copy all hooks from WORKDIR into debian/, hooks are not 
> generated 
>          for t in pre post 
>          do 
>                  for a in inst rm 
> -- 
> 2.23.0 
>
>

[-- Attachment #1.2: Type: text/html, Size: 10928 bytes --]

Re: [PATCH 4/5] doc: write something about "debianize"

[Henning Schild]

On Fri, 1 Nov 2019 00:08:35 -0700
<chombourger@gmail.com> wrote:

> On Tuesday, October 29, 2019 at 5:44:28 PM UTC+1, Henning Schild
> wrote:
> >
> > From: Henning Schild <hennin...@siemens.com <javascript:>> 
> >
> > Also add some comments to the implementation and extend the example 
> > recipe with some pre- and post-processing. 
> >
> > Signed-off-by: Henning Schild <hennin...@siemens.com <javascript:>> 
> > --- 
> >  doc/user_manual.md                            | 19
> > ++++++++++++++++++- .../recipes-app/samefile/samefile_2.14.bb     |
> > 14 ++++++++++++++ meta/classes/debianize.bbclass                |
> > 6 ++++++ 3 files changed, 38 insertions(+), 1 deletion(-) 
> >
> > diff --git a/doc/user_manual.md b/doc/user_manual.md 
> > index 34f50df..7c23fcc 100644 
> > --- a/doc/user_manual.md 
> > +++ b/doc/user_manual.md 
> > @@ -583,7 +583,7 @@ Isar currently supports two ways of creating
> > custom packages. 
> >   
> >  The `deb` packages are built using `dpkg-buildpackage`, so the
> > sources should contain the `debian` directory with necessary meta
> > information. This way is the default way of adding software that
> > needs to be compiled from source. The bbclass for this approach is
> > called `dpkg`. 
> > -**NOTE:** If the sources do not contain a `debian` directory your
> > recipe can fetch, create, or ship that. 
> > +**NOTE:** If the sources do not contain a `debian` directory your
> > recipe can fetch, create, or ship that. You might want to read the
> > the next section before returning here. 
> >   
> >  This is also what you do if you want to rebuild/modify an upstream 
> > package. 
> >  Isar does understand `SRC_URI` entries starting with "apt://". For
> > an example 
> > @@ -624,6 +624,23 @@ This approach prevents duplication of the
> > license files in different packages. 
> >   
> >  The last line in the example above adds recipe to the Isar work
> > chain. 
> > +### Compilation of sources missing the debian/-directory 
> > + 
> > +The `debian` directory contains meta information on how to build a 
> > package from source. This is roughly speaking "configure",
> > "compile", "install" all described in a Debian-specific way. 
> > +Isar expects your sources to contain the `debian` folder and the
> > above steps need to be described in it, not in a task in a recipe. 
> > + 
> > +So once you have sources you always need to combine them with a
> > `debian` folder before Isar can build a package for you. 
> > +You might be able to find a debianization for a component on the 
> > internet, i.e. Ubuntu does package an open source component while
> > Debian does not. Your recipe could download the `debian` folder
> > from Ubuntu and the sources from the open source project. 
> > + 
> > +You can write it yourself, which can be pretty easy but requires a
> > bit of studying.
> > <https://www.debian.org/doc/debian-policy/index.html> 
> > + 
> > +Isar does actually contain a helper that aims to "debianize"
> > sources for your. If your package uses a build-system that Debian
> > knows and follows the well known "configure", "compile", "install"
> > scheme that debianization might just fit your needs without reading
> > Debian manuals. +If it does not fully fit your needs, it probably
> > gives you a good starting point for your manual tuning. 
> > + 
> > +The shell function `deb_debianize` creates a `debian` folder. But
> > it will not overwrite files that already are in WORKDIR. So you can
> > either just call it to fully generate the `debian` folder. Or you
> > combine it with pre-existing parts. 
> > + 
> > +Have a look at meta-isar/recipes-app/samefile/samefile_2.14.bb and 
> > meta/classes/debianize.bbclass for an example and the
> > implementation. 
> > + 
> >  ### Packages without source 
> >   
> >  If your customization is not about compiling from source there is
> > a second way of creating `deb` packages. That way can be used for
> > cases like: diff --git
> > a/meta-isar/recipes-app/samefile/samefile_2.14.bb
> > b/meta-isar/recipes-app/samefile/samefile_2.14.bb index
> > a24d574..0ace85a 100644 ---
> > a/meta-isar/recipes-app/samefile/samefile_2.14.bb +++
> > b/meta-isar/recipes-app/samefile/samefile_2.14.bb @@ -12,5 +12,19
> > @@ SRC_URI[md5sum] = "0b438249f3549f18b49cbb49b0473f70" 
> >  do_prepare_build[cleandirs] += "${S}/debian" 
> >  do_prepare_build() { 
> > +    # you could also create parts of your debianization before 
> > +    # pre-exisiting files will not be recreated 
> > +    # 42 is not actually valid ... this is just a stupid example 
> > +    echo 42 > "${WORKDIR}"/compat 
> > + 
> > +    # this step creates everything dpkg-buildpackage needs, compat
> > will not be 
> > +    # overwritten 
> >      deb_debianize 
> > + 
> >  
> 
> do we want to "read compat <${S}/compat" before we overwrite it just
> to make sure it is still 42? and bbfatal if it isn't?
> in other words is this recipe more of a test or of an example?
> (either way is fine by me)

It is an example that is abused for partial testing. Good testing would
make it a bad example, so i will not change it.
If we ever get a testing layer, thorough testing of debianization would
be something to potentially add there.

Henning

> 
> > +    # now if you are not happy you can post-process the result 
> > +    # now let us "repair" the compat 
> > +    echo 9 > "${S}/debian/compat" 
> > +    # compat happens to be used by this step, so do that again 
> > +    # see debianize.bbclass 
> > +    deb_create_control 
> >  } 
> > diff --git a/meta/classes/debianize.bbclass 
> > b/meta/classes/debianize.bbclass 
> > index fb9c234..e1c79ab 100644 
> > --- a/meta/classes/debianize.bbclass 
> > +++ b/meta/classes/debianize.bbclass 
> > @@ -76,23 +76,29 @@ EOF 
> >  } 
> >   
> >  deb_debianize() { 
> > +        # create the compat-file if there is no file with that
> > name in WORKDIR 
> >          if [ -f ${WORKDIR}/compat ]; then 
> >                  install -v -m 644 ${WORKDIR}/compat
> > ${S}/debian/compat else 
> >                  deb_create_compat 
> >          fi 
> > +        # create the control-file if there is no contro-file in
> > WORKDIR if [ -f ${WORKDIR}/control ]; then 
> >                  install -v -m 644 ${WORKDIR}/control
> > ${S}/debian/control else 
> >                  deb_create_control 
> >          fi 
> > +        # create rules if WORKDIR does not contain a rules-file 
> >          if [ -f ${WORKDIR}/rules ]; then 
> >                  install -v -m 755 ${WORKDIR}/rules
> > ${S}/debian/rules else 
> >                  deb_create_rules 
> >          fi 
> > +        # prepend a changelog-entry unless an existing changelog
> > file already 
> > +        # contains an entry with CHANGELOG_V 
> >          deb_add_changelog 
> >   
> > +        # copy all hooks from WORKDIR into debian/, hooks are not 
> > generated 
> >          for t in pre post 
> >          do 
> >                  for a in inst rm 
> > -- 
> > 2.23.0 
> >
> >  
> 

Re: [PATCH 4/5] doc: write something about "debianize"

[Jan Kiszka]

On 29.10.19 17:44, Henning Schild wrote:
> From: Henning Schild <henning.schild@siemens.com>
> 
> Also add some comments to the implementation and extend the example
> recipe with some pre- and post-processing.
> 
> Signed-off-by: Henning Schild <henning.schild@siemens.com>
> ---
>  doc/user_manual.md                            | 19 ++++++++++++++++++-
>  .../recipes-app/samefile/samefile_2.14.bb     | 14 ++++++++++++++
>  meta/classes/debianize.bbclass                |  6 ++++++
>  3 files changed, 38 insertions(+), 1 deletion(-)
> 
> diff --git a/doc/user_manual.md b/doc/user_manual.md
> index 34f50df..7c23fcc 100644
> --- a/doc/user_manual.md
> +++ b/doc/user_manual.md
> @@ -583,7 +583,7 @@ Isar currently supports two ways of creating custom packages.
>  
>  The `deb` packages are built using `dpkg-buildpackage`, so the sources should contain the `debian` directory with necessary meta information. This way is the default way of adding software that needs to be compiled from source. The bbclass for this approach is called `dpkg`.
>  
> -**NOTE:** If the sources do not contain a `debian` directory your recipe can fetch, create, or ship that.
> +**NOTE:** If the sources do not contain a `debian` directory your recipe can fetch, create, or ship that. You might want to read the the next section before returning here.
>  
>  This is also what you do if you want to rebuild/modify an upstream package.
>  Isar does understand `SRC_URI` entries starting with "apt://". For an example
> @@ -624,6 +624,23 @@ This approach prevents duplication of the license files in different packages.
>  
>  The last line in the example above adds recipe to the Isar work chain.
>  
> +### Compilation of sources missing the debian/-directory
> +
> +The `debian` directory contains meta information on how to build a package from source. This is roughly speaking "configure", "compile", "install" all described in a Debian-specific way.
> +Isar expects your sources to contain the `debian` folder and the above steps need to be described in it, not in a task in a recipe.
> +
> +So once you have sources you always need to combine them with a `debian` folder before Isar can build a package for you.
> +You might be able to find a debianization for a component on the internet, i.e. Ubuntu does package an open source component while Debian does not. Your recipe could download the `debian` folder from Ubuntu and the sources from the open source project.
> +
> +You can write it yourself, which can be pretty easy but requires a bit of studying. <https://www.debian.org/doc/debian-policy/index.html>
> +
> +Isar does actually contain a helper that aims to "debianize" sources for your. If your package uses a build-system that Debian knows and follows the well known "configure", "compile", "install" scheme that debianization might just fit your needs without reading Debian manuals.
> +If it does not fully fit your needs, it probably gives you a good starting point for your manual tuning.
> +
> +The shell function `deb_debianize` creates a `debian` folder. But it will not overwrite files that already are in WORKDIR. So you can either just call it to fully generate the `debian` folder. Or you combine it with pre-existing parts.
> +
> +Have a look at meta-isar/recipes-app/samefile/samefile_2.14.bb and meta/classes/debianize.bbclass for an example and the implementation.
> +
>  ### Packages without source
>  
>  If your customization is not about compiling from source there is a second way of creating `deb` packages. That way can be used for cases like:
> diff --git a/meta-isar/recipes-app/samefile/samefile_2.14.bb b/meta-isar/recipes-app/samefile/samefile_2.14.bb
> index a24d574..0ace85a 100644
> --- a/meta-isar/recipes-app/samefile/samefile_2.14.bb
> +++ b/meta-isar/recipes-app/samefile/samefile_2.14.bb
> @@ -12,5 +12,19 @@ SRC_URI[md5sum] = "0b438249f3549f18b49cbb49b0473f70"
>  
>  do_prepare_build[cleandirs] += "${S}/debian"
>  do_prepare_build() {
> +    # you could also create parts of your debianization before
> +    # pre-exisiting files will not be recreated

Let's write sentences and separate them.

> +    # 42 is not actually valid ... this is just a stupid example
> +    echo 42 > "${WORKDIR}"/compat

Don't we have a better example than writing something invalid which only
needs to be reverted later on?

> +
> +    # this step creates everything dpkg-buildpackage needs, compat will not be
> +    # overwritten
>      deb_debianize
> +
> +    # now if you are not happy you can post-process the result
> +    # now let us "repair" the compat
> +    echo 9 > "${S}/debian/compat"

Better append something to the generated file, maybe change the package
category or so.

> +    # compat happens to be used by this step, so do that again
> +    # see debianize.bbclass
> +    deb_create_control
>  }
> diff --git a/meta/classes/debianize.bbclass b/meta/classes/debianize.bbclass
> index fb9c234..e1c79ab 100644
> --- a/meta/classes/debianize.bbclass
> +++ b/meta/classes/debianize.bbclass
> @@ -76,23 +76,29 @@ EOF
>  }
>  
>  deb_debianize() {
> +	# create the compat-file if there is no file with that name in WORKDIR
>  	if [ -f ${WORKDIR}/compat ]; then
>  		install -v -m 644 ${WORKDIR}/compat ${S}/debian/compat
>  	else
>  		deb_create_compat
>  	fi
> +	# create the control-file if there is no contro-file in WORKDIR

control file

>  	if [ -f ${WORKDIR}/control ]; then
>  		install -v -m 644 ${WORKDIR}/control ${S}/debian/control
>  	else
>  		deb_create_control
>  	fi
> +	# create rules if WORKDIR does not contain a rules-file
>  	if [ -f ${WORKDIR}/rules ]; then
>  		install -v -m 755 ${WORKDIR}/rules ${S}/debian/rules
>  	else
>  		deb_create_rules
>  	fi
> +	# prepend a changelog-entry unless an existing changelog file already
> +	# contains an entry with CHANGELOG_V
>  	deb_add_changelog
>  
> +	# copy all hooks from WORKDIR into debian/, hooks are not generated
>  	for t in pre post
>  	do
>  		for a in inst rm
> 

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH 4/5] doc: write something about "debianize"

[Henning Schild]

Am Wed, 6 Nov 2019 17:26:39 +0100
schrieb Jan Kiszka <jan.kiszka@siemens.com>:

> On 29.10.19 17:44, Henning Schild wrote:
> > From: Henning Schild <henning.schild@siemens.com>
> > 
> > Also add some comments to the implementation and extend the example
> > recipe with some pre- and post-processing.
> > 
> > Signed-off-by: Henning Schild <henning.schild@siemens.com>
> > ---
> >  doc/user_manual.md                            | 19
> > ++++++++++++++++++- .../recipes-app/samefile/samefile_2.14.bb     |
> > 14 ++++++++++++++ meta/classes/debianize.bbclass                |
> > 6 ++++++ 3 files changed, 38 insertions(+), 1 deletion(-)
> > 
> > diff --git a/doc/user_manual.md b/doc/user_manual.md
> > index 34f50df..7c23fcc 100644
> > --- a/doc/user_manual.md
> > +++ b/doc/user_manual.md
> > @@ -583,7 +583,7 @@ Isar currently supports two ways of creating
> > custom packages. 
> >  The `deb` packages are built using `dpkg-buildpackage`, so the
> > sources should contain the `debian` directory with necessary meta
> > information. This way is the default way of adding software that
> > needs to be compiled from source. The bbclass for this approach is
> > called `dpkg`. -**NOTE:** If the sources do not contain a `debian`
> > directory your recipe can fetch, create, or ship that. +**NOTE:**
> > If the sources do not contain a `debian` directory your recipe can
> > fetch, create, or ship that. You might want to read the the next
> > section before returning here. This is also what you do if you want
> > to rebuild/modify an upstream package. Isar does understand
> > `SRC_URI` entries starting with "apt://". For an example @@ -624,6
> > +624,23 @@ This approach prevents duplication of the license files
> > in different packages. The last line in the example above adds
> > recipe to the Isar work chain. +### Compilation of sources missing
> > the debian/-directory + +The `debian` directory contains meta
> > information on how to build a package from source. This is roughly
> > speaking "configure", "compile", "install" all described in a
> > Debian-specific way. +Isar expects your sources to contain the
> > `debian` folder and the above steps need to be described in it, not
> > in a task in a recipe. + +So once you have sources you always need
> > to combine them with a `debian` folder before Isar can build a
> > package for you. +You might be able to find a debianization for a
> > component on the internet, i.e. Ubuntu does package an open source
> > component while Debian does not. Your recipe could download the
> > `debian` folder from Ubuntu and the sources from the open source
> > project. + +You can write it yourself, which can be pretty easy but
> > requires a bit of studying.
> > <https://www.debian.org/doc/debian-policy/index.html> + +Isar does
> > actually contain a helper that aims to "debianize" sources for
> > your. If your package uses a build-system that Debian knows and
> > follows the well known "configure", "compile", "install" scheme
> > that debianization might just fit your needs without reading Debian
> > manuals. +If it does not fully fit your needs, it probably gives
> > you a good starting point for your manual tuning. + +The shell
> > function `deb_debianize` creates a `debian` folder. But it will not
> > overwrite files that already are in WORKDIR. So you can either just
> > call it to fully generate the `debian` folder. Or you combine it
> > with pre-existing parts. + +Have a look at
> > meta-isar/recipes-app/samefile/samefile_2.14.bb and
> > meta/classes/debianize.bbclass for an example and the
> > implementation. + ### Packages without source If your customization
> > is not about compiling from source there is a second way of
> > creating `deb` packages. That way can be used for cases like: diff
> > --git a/meta-isar/recipes-app/samefile/samefile_2.14.bb
> > b/meta-isar/recipes-app/samefile/samefile_2.14.bb index
> > a24d574..0ace85a 100644 ---
> > a/meta-isar/recipes-app/samefile/samefile_2.14.bb +++
> > b/meta-isar/recipes-app/samefile/samefile_2.14.bb @@ -12,5 +12,19
> > @@ SRC_URI[md5sum] = "0b438249f3549f18b49cbb49b0473f70"
> > do_prepare_build[cleandirs] += "${S}/debian" do_prepare_build() {
> > +    # you could also create parts of your debianization before
> > +    # pre-exisiting files will not be recreated  
> 
> Let's write sentences and separate them.
> 
> > +    # 42 is not actually valid ... this is just a stupid example
> > +    echo 42 > "${WORKDIR}"/compat  
> 
> Don't we have a better example than writing something invalid which
> only needs to be reverted later on?

Though about it and went for a hook and an old changelog.

> > +
> > +    # this step creates everything dpkg-buildpackage needs, compat
> > will not be
> > +    # overwritten
> >      deb_debianize
> > +
> > +    # now if you are not happy you can post-process the result
> > +    # now let us "repair" the compat
> > +    echo 9 > "${S}/debian/compat"  
> 
> Better append something to the generated file, maybe change the
> package category or so.

Good idea, makes a valuable example indeed!

Henning

> > +    # compat happens to be used by this step, so do that again
> > +    # see debianize.bbclass
> > +    deb_create_control
> >  }
> > diff --git a/meta/classes/debianize.bbclass
> > b/meta/classes/debianize.bbclass index fb9c234..e1c79ab 100644
> > --- a/meta/classes/debianize.bbclass
> > +++ b/meta/classes/debianize.bbclass
> > @@ -76,23 +76,29 @@ EOF
> >  }
> >  
> >  deb_debianize() {
> > +	# create the compat-file if there is no file with that
> > name in WORKDIR if [ -f ${WORKDIR}/compat ]; then
> >  		install -v -m 644 ${WORKDIR}/compat
> > ${S}/debian/compat else
> >  		deb_create_compat
> >  	fi
> > +	# create the control-file if there is no contro-file in
> > WORKDIR  
> 
> control file
> 
> >  	if [ -f ${WORKDIR}/control ]; then
> >  		install -v -m 644 ${WORKDIR}/control
> > ${S}/debian/control else
> >  		deb_create_control
> >  	fi
> > +	# create rules if WORKDIR does not contain a rules-file
> >  	if [ -f ${WORKDIR}/rules ]; then
> >  		install -v -m 755 ${WORKDIR}/rules
> > ${S}/debian/rules else
> >  		deb_create_rules
> >  	fi
> > +	# prepend a changelog-entry unless an existing changelog
> > file already
> > +	# contains an entry with CHANGELOG_V
> >  	deb_add_changelog
> >  
> > +	# copy all hooks from WORKDIR into debian/, hooks are not
> > generated for t in pre post
> >  	do
> >  		for a in inst rm
> >   
> 
> Jan
> 

[PATCH 5/5] CI: add "samefile" to our example config and CI build

[Henning Schild]

From: Henning Schild <henning.schild@siemens.com>

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta-isar/conf/local.conf.sample | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample
index 7715ce8..280e007 100644
--- a/meta-isar/conf/local.conf.sample
+++ b/meta-isar/conf/local.conf.sample
@@ -166,7 +166,7 @@ CONF_VERSION = "1"
 
 #
 # The default list of extra packages to be installed.
-IMAGE_INSTALL = "hello-isar example-raw example-module-${KERNEL_NAME} enable-fsck isar-exclude-docs"
+IMAGE_INSTALL = "hello-isar example-raw example-module-${KERNEL_NAME} enable-fsck isar-exclude-docs samefile"
 ## "cache_base_repo" relies on the cache ...
 #IMAGE_INSTALL += "isar-disable-apt-cache"
 ## uses apt:// SRC_URI, which does not work with the cache
-- 
2.23.0