[PATCH 0/4] miscellaneous (minor) fixes

[PATCH 0/4] miscellaneous (minor) fixes

[Cedric Hombourger]

The following minor fixes are being submitted for review and inclusion to "next"

Cedric Hombourger (4):
  wic-img: do not leave temporary files behind
  isar-bootstrap: wipe out previously created rootfs with
    --one-file-system
  base-apt-helper: fixed a few typos found in comments
  patch: suffix patch files copied to .applied_patches with "~"

 meta/classes/base-apt-helper.bbclass                | 4 ++--
 meta/classes/patch.bbclass                          | 2 +-
 meta/classes/wic-img.bbclass                        | 8 ++++++--
 meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 2 +-
 4 files changed, 10 insertions(+), 6 deletions(-)

-- 
2.20.1

[PATCH 1/4] wic-img: do not leave temporary files behind

[Cedric Hombourger]

We do not seem to make any use of the timestamped wic images being
generated and worse we are leaving them in the buildchroot tmp folder.
"mv" the generated files instead of "cp"'ing them to the deploy
directory.

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/classes/wic-img.bbclass | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/meta/classes/wic-img.bbclass b/meta/classes/wic-img.bbclass
index 795640f..24c3b65 100644
--- a/meta/classes/wic-img.bbclass
+++ b/meta/classes/wic-img.bbclass
@@ -155,8 +155,12 @@ EOSUDO
             -e ${IMAGE_BASENAME} ${WIC_CREATE_EXTRA_ARGS}
     sudo chown -R $(stat -c "%U" ${ISARROOT}) ${ISARROOT}/meta ${ISARROOT}/meta-isar ${ISARROOT}/scripts || true
     WIC_DIRECT=$(ls -t -1 ${BUILDCHROOT_DIR}/$WICTMP/${IMAGE_FULLNAME}.wic/*.direct | head -1)
-    cp -f ${WIC_DIRECT} ${WIC_IMAGE_FILE}
-    cp -f ${WIC_DIRECT}.bmap ${WIC_IMAGE_FILE}.bmap
+    sudo chown $(stat -c "%U:%G" ${ISARROOT}) ${WIC_DIRECT} ${WIC_DIRECT}.bmap
+    sudo mv -f ${WIC_DIRECT} ${WIC_IMAGE_FILE}
+    sudo mv -f ${WIC_DIRECT}.bmap ${WIC_IMAGE_FILE}.bmap
+    sudo rmdir --ignore-fail-on-non-empty \
+        ${BUILDCHROOT_DIR}/${WICTMP}/${IMAGE_FULLNAME}.wic \
+        ${BUILDCHROOT_DIR}/${WICTMP}
 }
 
 do_wic_image[file-checksums] += "${WKS_FILE_CHECKSUM}"
-- 
2.20.1

[PATCH 2/4] isar-bootstrap: wipe out previously created rootfs with --one-file-system

[Cedric Hombourger]

As a safety precaution and to prevent the accidental deletion of files
from dangling bind mounts, remove the previously created rootfs with
rm -rf --one-file-system

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
index 1a58471..0fb033b 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -245,7 +245,7 @@ isar_bootstrap() {
     sudo -E -s <<'EOSUDO'
         set -e
         if [ ! -e "${DEPLOY_ISAR_BOOTSTRAP}" ]; then
-            rm -rf "${ROOTFSDIR}"
+            rm -rf --one-file-system "${ROOTFSDIR}"
 
             if [ ${IS_HOST} ]; then
                 ${DEBOOTSTRAP} $debootstrap_args \
-- 
2.20.1

[PATCH 3/4] base-apt-helper: fixed a few typos found in comments

[Cedric Hombourger]

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/classes/base-apt-helper.bbclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/base-apt-helper.bbclass b/meta/classes/base-apt-helper.bbclass
index 81df45a..31b2832 100644
--- a/meta/classes/base-apt-helper.bbclass
+++ b/meta/classes/base-apt-helper.bbclass
@@ -21,11 +21,11 @@ populate_base_apt() {
     find $search_dir -name '*.deb' | while read package; do
         # NOTE: due to packages stored by reprepro are not modified, we can
         # use search by filename to check if package is already in repo. In
-        # addition, m5sums could be compared to ensure, that package is the
+        # addition, md5sums are compared to ensure that the package is the
         # same and should not be overwritten. This method is easier and more
         # robust than querying reprepro by name.
 
-        # Check if this package is taken from Isar-apt, if so - ingore it.
+        # Check if this package is taken from Isar-apt, if so - ignore it.
         base_name=${package##*/}
         isar_package=$(find ${REPO_ISAR_DIR}/${DISTRO} -name $base_name)
         if [ -n "$isar_package" ]; then
-- 
2.20.1

[PATCH 4/4] patch: suffix patch files copied to .applied_patches with "~"

[Cedric Hombourger]

When patches are applied to a source Debian package, patches specified in
SRC_URI get copied to ${S}/.applied_patches (as intended) but dpkg-source
then believes the source tree was modified and exits with an error. Make
dpkg-source ignore patch files that were copied by this class by appending
a "~" suffix to them.

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/classes/patch.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/patch.bbclass b/meta/classes/patch.bbclass
index c5ba463..1d8041f 100644
--- a/meta/classes/patch.bbclass
+++ b/meta/classes/patch.bbclass
@@ -54,7 +54,7 @@ python do_patch() {
 
             cmds = open(applied_patches_dir + ".patch-commands", "a")
 
-            patch_file = applied_patches_dir + path
+            patch_file = applied_patches_dir + path + "~"
             bb.utils.mkdirhier(os.path.dirname(patch_file))
             shutil.copyfile(workdir + path, patch_file)
 
-- 
2.20.1

Re: [PATCH 4/4] patch: suffix patch files copied to .applied_patches with "~"

[Henning Schild]

I think that whole patching keeps the files so we can roll back and i
think Jan wanted to somehow continue working on it using quilt.

Renaming that file might break that, did you check?

I guess we could introduce a test for parts of that by adding a patch
to the hello recipe.

Henning

Am Mon, 21 Oct 2019 11:34:54 +0200
schrieb Cedric Hombourger <Cedric_Hombourger@mentor.com>:

> When patches are applied to a source Debian package, patches
> specified in SRC_URI get copied to ${S}/.applied_patches (as
> intended) but dpkg-source then believes the source tree was modified
> and exits with an error. Make dpkg-source ignore patch files that
> were copied by this class by appending a "~" suffix to them.
> 
> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
> ---
>  meta/classes/patch.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/classes/patch.bbclass b/meta/classes/patch.bbclass
> index c5ba463..1d8041f 100644
> --- a/meta/classes/patch.bbclass
> +++ b/meta/classes/patch.bbclass
> @@ -54,7 +54,7 @@ python do_patch() {
>  
>              cmds = open(applied_patches_dir + ".patch-commands", "a")
>  
> -            patch_file = applied_patches_dir + path
> +            patch_file = applied_patches_dir + path + "~"
>              bb.utils.mkdirhier(os.path.dirname(patch_file))
>              shutil.copyfile(workdir + path, patch_file)
>  

Re: [PATCH 4/4] patch: suffix patch files copied to .applied_patches with "~"

[Cedric Hombourger]

Hi Henning

On 10/21/2019 1:26 PM, Henning Schild wrote:
> I think that whole patching keeps the files so we can roll back and i
> think Jan wanted to somehow continue working on it using quilt.

Ok the last I heard from him is that he felt that the approach he had 
implemented and upstream'ed was far easier than having to implement 
something ala Yocto. I guess we can let him comment?

>
> Renaming that file might break that, did you check?

yes, no issues observed on my end with e.g. the linux-mainline kernel 
recipe on the de0-nano machine. The .patch_commands file carries the 
full "patch" command and the expected filename (suffixed with ~). Isar 
simply appends -R to the patch command line when it needs to revert 
previously applied patches.

> I guess we could introduce a test for parts of that by adding a patch
> to the hello recipe.

We sure could though the linux-mainline recipe is already testing this.

>
> Henning
>
> Am Mon, 21 Oct 2019 11:34:54 +0200
> schrieb Cedric Hombourger <Cedric_Hombourger@mentor.com>:
>
>> When patches are applied to a source Debian package, patches
>> specified in SRC_URI get copied to ${S}/.applied_patches (as
>> intended) but dpkg-source then believes the source tree was modified
>> and exits with an error. Make dpkg-source ignore patch files that
>> were copied by this class by appending a "~" suffix to them.
>>
>> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
>> ---
>>   meta/classes/patch.bbclass | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/meta/classes/patch.bbclass b/meta/classes/patch.bbclass
>> index c5ba463..1d8041f 100644
>> --- a/meta/classes/patch.bbclass
>> +++ b/meta/classes/patch.bbclass
>> @@ -54,7 +54,7 @@ python do_patch() {
>>   
>>               cmds = open(applied_patches_dir + ".patch-commands", "a")
>>   
>> -            patch_file = applied_patches_dir + path
>> +            patch_file = applied_patches_dir + path + "~"
>>               bb.utils.mkdirhier(os.path.dirname(patch_file))
>>               shutil.copyfile(workdir + path, patch_file)
>>   

Re: [PATCH 4/4] patch: suffix patch files copied to .applied_patches with "~"

[Henning Schild]

Am Mon, 21 Oct 2019 13:36:25 +0200
schrieb Cedric Hombourger <cedric_hombourger@mentor.com>:

> Hi Henning
> 
> On 10/21/2019 1:26 PM, Henning Schild wrote:
> > I think that whole patching keeps the files so we can roll back and
> > i think Jan wanted to somehow continue working on it using quilt.  
> 
> Ok the last I heard from him is that he felt that the approach he had 
> implemented and upstream'ed was far easier than having to implement 
> something ala Yocto. I guess we can let him comment?
> 
> >
> > Renaming that file might break that, did you check?  
> 
> yes, no issues observed on my end with e.g. the linux-mainline kernel 
> recipe on the de0-nano machine. The .patch_commands file carries the 
> full "patch" command and the expected filename (suffixed with ~).
> Isar simply appends -R to the patch command line when it needs to
> revert previously applied patches.
> 
> > I guess we could introduce a test for parts of that by adding a
> > patch to the hello recipe.  
> 
> We sure could though the linux-mainline recipe is already testing
> this.

I was talking about the combination apt://-URI + patch(es)

Henning

> >
> > Henning
> >
> > Am Mon, 21 Oct 2019 11:34:54 +0200
> > schrieb Cedric Hombourger <Cedric_Hombourger@mentor.com>:
> >  
> >> When patches are applied to a source Debian package, patches
> >> specified in SRC_URI get copied to ${S}/.applied_patches (as
> >> intended) but dpkg-source then believes the source tree was
> >> modified and exits with an error. Make dpkg-source ignore patch
> >> files that were copied by this class by appending a "~" suffix to
> >> them.
> >>
> >> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
> >> ---
> >>   meta/classes/patch.bbclass | 2 +-
> >>   1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/meta/classes/patch.bbclass
> >> b/meta/classes/patch.bbclass index c5ba463..1d8041f 100644
> >> --- a/meta/classes/patch.bbclass
> >> +++ b/meta/classes/patch.bbclass
> >> @@ -54,7 +54,7 @@ python do_patch() {
> >>   
> >>               cmds = open(applied_patches_dir + ".patch-commands",
> >> "a") 
> >> -            patch_file = applied_patches_dir + path
> >> +            patch_file = applied_patches_dir + path + "~"
> >>               bb.utils.mkdirhier(os.path.dirname(patch_file))
> >>               shutil.copyfile(workdir + path, patch_file)
> >>     

Re: [PATCH 4/4] patch: suffix patch files copied to .applied_patches with "~"

[Cedric Hombourger]

On 10/21/2019 2:43 PM, Henning Schild wrote:
> Am Mon, 21 Oct 2019 13:36:25 +0200
> schrieb Cedric Hombourger <cedric_hombourger@mentor.com>:
>
>> Hi Henning
>>
>> On 10/21/2019 1:26 PM, Henning Schild wrote:
>>> I think that whole patching keeps the files so we can roll back and
>>> i think Jan wanted to somehow continue working on it using quilt.
>> Ok the last I heard from him is that he felt that the approach he had
>> implemented and upstream'ed was far easier than having to implement
>> something ala Yocto. I guess we can let him comment?
>>
>>> Renaming that file might break that, did you check?
>> yes, no issues observed on my end with e.g. the linux-mainline kernel
>> recipe on the de0-nano machine. The .patch_commands file carries the
>> full "patch" command and the expected filename (suffixed with ~).
>> Isar simply appends -R to the patch command line when it needs to
>> revert previously applied patches.
>>
>>> I guess we could introduce a test for parts of that by adding a
>>> patch to the hello recipe.
>> We sure could though the linux-mainline recipe is already testing
>> this.
> I was talking about the combination apt://-URI + patch(es)

Copy that. I will figure out something

>
> Henning
>
>>> Henning
>>>
>>> Am Mon, 21 Oct 2019 11:34:54 +0200
>>> schrieb Cedric Hombourger <Cedric_Hombourger@mentor.com>:
>>>   
>>>> When patches are applied to a source Debian package, patches
>>>> specified in SRC_URI get copied to ${S}/.applied_patches (as
>>>> intended) but dpkg-source then believes the source tree was
>>>> modified and exits with an error. Make dpkg-source ignore patch
>>>> files that were copied by this class by appending a "~" suffix to
>>>> them.
>>>>
>>>> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
>>>> ---
>>>>    meta/classes/patch.bbclass | 2 +-
>>>>    1 file changed, 1 insertion(+), 1 deletion(-)
>>>>
>>>> diff --git a/meta/classes/patch.bbclass
>>>> b/meta/classes/patch.bbclass index c5ba463..1d8041f 100644
>>>> --- a/meta/classes/patch.bbclass
>>>> +++ b/meta/classes/patch.bbclass
>>>> @@ -54,7 +54,7 @@ python do_patch() {
>>>>    
>>>>                cmds = open(applied_patches_dir + ".patch-commands",
>>>> "a")
>>>> -            patch_file = applied_patches_dir + path
>>>> +            patch_file = applied_patches_dir + path + "~"
>>>>                bb.utils.mkdirhier(os.path.dirname(patch_file))
>>>>                shutil.copyfile(workdir + path, patch_file)
>>>>      

[PATCH v2 0/5] miscellaneous (minor) fixes

[Cedric Hombourger]

The following minor fixes are being submitted for review and inclusion to "next"

Changes compared to v1:
  the ".patch-commands" file should also be suffixed with "~"
  patch the hello package to test patch.bbclass from a dpkg recipe

Cedric Hombourger (5):
  wic-img: do not leave temporary files behind
  isar-bootstrap: wipe out previously created rootfs with
    --one-file-system
  base-apt-helper: fixed a few typos found in comments
  patch: suffix patch files copied to .applied_patches with "~"
  hello: add a sample patch to exercise patch.bbclass against dpkg
    recipes

 .../files/add-built-by-isar-to-version.patch  | 22 +++++++++++++++++++
 meta-isar/recipes-app/hello/hello.inc         |  3 +++
 meta/classes/base-apt-helper.bbclass          |  4 ++--
 meta/classes/patch.bbclass                    |  6 ++---
 meta/classes/wic-img.bbclass                  |  8 +++++--
 .../isar-bootstrap/isar-bootstrap.inc         |  2 +-
 6 files changed, 37 insertions(+), 8 deletions(-)
 create mode 100644 meta-isar/recipes-app/hello/files/add-built-by-isar-to-version.patch

-- 
2.20.1

[PATCH v2 1/5] wic-img: do not leave temporary files behind

[Cedric Hombourger]

We do not seem to make any use of the timestamped wic images being
generated and worse we are leaving them in the buildchroot tmp folder.
"mv" the generated files instead of "cp"'ing them to the deploy
directory.

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/classes/wic-img.bbclass | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/meta/classes/wic-img.bbclass b/meta/classes/wic-img.bbclass
index 795640f..24c3b65 100644
--- a/meta/classes/wic-img.bbclass
+++ b/meta/classes/wic-img.bbclass
@@ -155,8 +155,12 @@ EOSUDO
             -e ${IMAGE_BASENAME} ${WIC_CREATE_EXTRA_ARGS}
     sudo chown -R $(stat -c "%U" ${ISARROOT}) ${ISARROOT}/meta ${ISARROOT}/meta-isar ${ISARROOT}/scripts || true
     WIC_DIRECT=$(ls -t -1 ${BUILDCHROOT_DIR}/$WICTMP/${IMAGE_FULLNAME}.wic/*.direct | head -1)
-    cp -f ${WIC_DIRECT} ${WIC_IMAGE_FILE}
-    cp -f ${WIC_DIRECT}.bmap ${WIC_IMAGE_FILE}.bmap
+    sudo chown $(stat -c "%U:%G" ${ISARROOT}) ${WIC_DIRECT} ${WIC_DIRECT}.bmap
+    sudo mv -f ${WIC_DIRECT} ${WIC_IMAGE_FILE}
+    sudo mv -f ${WIC_DIRECT}.bmap ${WIC_IMAGE_FILE}.bmap
+    sudo rmdir --ignore-fail-on-non-empty \
+        ${BUILDCHROOT_DIR}/${WICTMP}/${IMAGE_FULLNAME}.wic \
+        ${BUILDCHROOT_DIR}/${WICTMP}
 }
 
 do_wic_image[file-checksums] += "${WKS_FILE_CHECKSUM}"
-- 
2.20.1

[PATCH v2 2/5] isar-bootstrap: wipe out previously created rootfs with --one-file-system

[Cedric Hombourger]

As a safety precaution and to prevent the accidental deletion of files
from dangling bind mounts, remove the previously created rootfs with
rm -rf --one-file-system

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
index 1a58471..0fb033b 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -245,7 +245,7 @@ isar_bootstrap() {
     sudo -E -s <<'EOSUDO'
         set -e
         if [ ! -e "${DEPLOY_ISAR_BOOTSTRAP}" ]; then
-            rm -rf "${ROOTFSDIR}"
+            rm -rf --one-file-system "${ROOTFSDIR}"
 
             if [ ${IS_HOST} ]; then
                 ${DEBOOTSTRAP} $debootstrap_args \
-- 
2.20.1

[PATCH v2 3/5] base-apt-helper: fixed a few typos found in comments

[Cedric Hombourger]

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/classes/base-apt-helper.bbclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/base-apt-helper.bbclass b/meta/classes/base-apt-helper.bbclass
index 81df45a..31b2832 100644
--- a/meta/classes/base-apt-helper.bbclass
+++ b/meta/classes/base-apt-helper.bbclass
@@ -21,11 +21,11 @@ populate_base_apt() {
     find $search_dir -name '*.deb' | while read package; do
         # NOTE: due to packages stored by reprepro are not modified, we can
         # use search by filename to check if package is already in repo. In
-        # addition, m5sums could be compared to ensure, that package is the
+        # addition, md5sums are compared to ensure that the package is the
         # same and should not be overwritten. This method is easier and more
         # robust than querying reprepro by name.
 
-        # Check if this package is taken from Isar-apt, if so - ingore it.
+        # Check if this package is taken from Isar-apt, if so - ignore it.
         base_name=${package##*/}
         isar_package=$(find ${REPO_ISAR_DIR}/${DISTRO} -name $base_name)
         if [ -n "$isar_package" ]; then
-- 
2.20.1

[PATCH v2 4/5] patch: suffix patch files copied to .applied_patches with "~"

[Cedric Hombourger]

When patches are applied to a source Debian package, patches specified in
SRC_URI get copied to ${S}/.applied_patches (as intended) but dpkg-source
then believes the source tree was modified and exits with an error. Make
dpkg-source ignore patch files that were copied by this class by appending
a "~" suffix to them.

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/classes/patch.bbclass | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/classes/patch.bbclass b/meta/classes/patch.bbclass
index c5ba463..e75d53c 100644
--- a/meta/classes/patch.bbclass
+++ b/meta/classes/patch.bbclass
@@ -8,7 +8,7 @@ def clean_applied_patches(applied_patches_dir):
     if not os.path.exists(applied_patches_dir):
         return
 
-    cmds_file = applied_patches_dir + ".patch-commands"
+    cmds_file = applied_patches_dir + ".patch-commands~"
     if os.path.exists(cmds_file):
         with open(cmds_file, "r") as cmds:
             patch_commands = cmds.readlines()
@@ -52,9 +52,9 @@ python do_patch() {
                 bb.utils.mkdirhier(applied_patches_dir)
                 applied_patches_dirs.append(applied_patches_dir)
 
-            cmds = open(applied_patches_dir + ".patch-commands", "a")
+            cmds = open(applied_patches_dir + ".patch-commands~", "a")
 
-            patch_file = applied_patches_dir + path
+            patch_file = applied_patches_dir + path + "~"
             bb.utils.mkdirhier(os.path.dirname(patch_file))
             shutil.copyfile(workdir + path, patch_file)
 
-- 
2.20.1

[PATCH v2 5/5] hello: add a sample patch to exercise patch.bbclass against dpkg recipes

[Cedric Hombourger]

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 .../files/add-built-by-isar-to-version.patch  | 22 +++++++++++++++++++
 meta-isar/recipes-app/hello/hello.inc         |  3 +++
 2 files changed, 25 insertions(+)
 create mode 100644 meta-isar/recipes-app/hello/files/add-built-by-isar-to-version.patch

diff --git a/meta-isar/recipes-app/hello/files/add-built-by-isar-to-version.patch b/meta-isar/recipes-app/hello/files/add-built-by-isar-to-version.patch
new file mode 100644
index 0000000..1ae879a
--- /dev/null
+++ b/meta-isar/recipes-app/hello/files/add-built-by-isar-to-version.patch
@@ -0,0 +1,22 @@
+diff -uNr hello-2.10.orig/debian/patches/add-built-by-isar-to-version.patch hello-2.10/debian/patches/add-built-by-isar-to-version.patch
+--- hello-2.10.orig/debian/patches/add-built-by-isar-to-version.patch	1969-12-31 19:00:00.000000000 -0500
++++ hello-2.10/debian/patches/add-built-by-isar-to-version.patch	2019-10-21 10:46:18.865268349 -0400
+@@ -0,0 +1,13 @@
++Index: hello-2.10/src/hello.c
++===================================================================
++--- hello-2.10.orig/src/hello.c
+++++ hello-2.10/src/hello.c
++@@ -169,7 +169,7 @@ Report bugs to: %s\n"), PACKAGE_BUGREPOR
++ static void
++ print_version (void)
++ {
++-  printf ("%s (%s) %s\n", PACKAGE, PACKAGE_NAME, VERSION);
+++  printf ("%s (%s) %s (built by Isar)\n", PACKAGE, PACKAGE_NAME, VERSION);
++   /* xgettext: no-wrap */
++   puts ("");
++ 
+diff -uNr hello-2.10.orig/debian/patches/series hello-2.10/debian/patches/series
+--- hello-2.10.orig/debian/patches/series	1969-12-31 19:00:00.000000000 -0500
++++ hello-2.10/debian/patches/series	2019-10-21 10:46:02.049662973 -0400
+@@ -0,0 +1 @@
++add-built-by-isar-to-version.patch
diff --git a/meta-isar/recipes-app/hello/hello.inc b/meta-isar/recipes-app/hello/hello.inc
index 747b96b..e59ea40 100644
--- a/meta-isar/recipes-app/hello/hello.inc
+++ b/meta-isar/recipes-app/hello/hello.inc
@@ -8,6 +8,9 @@ inherit dpkg
 # this will fetch and unpack the sources from upstream debian
 SRC_URI = "apt://${PN}=${PV}"
 
+# modify the output of the --version option to demonstrate how to patch upstream packages
+SRC_URI += "file://add-built-by-isar-to-version.patch"
+
 MAINTAINER = "isar-users <isar-users@googlegroups.com>"
 CHANGELOG_V = "${PV}-99+isar"
 
-- 
2.20.1

Re: [PATCH 4/4] patch: suffix patch files copied to .applied_patches with "~"

[Jan Kiszka]

On 21.10.19 13:36, Cedric Hombourger wrote:
> Hi Henning
> 
> On 10/21/2019 1:26 PM, Henning Schild wrote:
>> I think that whole patching keeps the files so we can roll back and i
>> think Jan wanted to somehow continue working on it using quilt.
> 
> Ok the last I heard from him is that he felt that the approach he had
> implemented and upstream'ed was far easier than having to implement
> something ala Yocto. I guess we can let him comment?

I was wrong, the current implementation is too broken. You can trigger
that by patching a tarball archive and then triggering an unpack
followed by a (re-)patch. That's why we need a mechanism with more
robust rollback support - i.e. OE quilt.

That said, this suffixing is Debian/Isar-specific and will "only" need
adjustment when porting the OE patch class over.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH 4/4] patch: suffix patch files copied to .applied_patches with "~"

[Cedric Hombourger]

On 10/25/2019 7:15 PM, Jan Kiszka wrote:
> On 21.10.19 13:36, Cedric Hombourger wrote:
>> Hi Henning
>>
>> On 10/21/2019 1:26 PM, Henning Schild wrote:
>>> I think that whole patching keeps the files so we can roll back and i
>>> think Jan wanted to somehow continue working on it using quilt.
>> Ok the last I heard from him is that he felt that the approach he had
>> implemented and upstream'ed was far easier than having to implement
>> something ala Yocto. I guess we can let him comment?
> I was wrong, the current implementation is too broken. You can trigger
> that by patching a tarball archive and then triggering an unpack
> followed by a (re-)patch. That's why we need a mechanism with more
> robust rollback support - i.e. OE quilt.

Ok I can drop this patch for now (no need to change something if we know 
it is broken).

Maybe I can work on this later (possibly after the work I started to 
rework our custom-kernel recipe so that it builds perf, status by the 
way is that the same set of kernel packages than what builddeb generates 
are created with hard-coded settings (for now), next steps are (1) use 
templates (2) generate perf packages). I do not have an ETA because I am 
working on this in parallel to some other activities - sorry for those 
of you waiting on this :)

>
> That said, this suffixing is Debian/Isar-specific and will "only" need
> adjustment when porting the OE patch class over.
>
> Jan
>

[PATCH v3 0/6] miscellaneous (minor) fixes

[Cedric Hombourger]

The following minor fixes are being submitted for review and inclusion to "next"

Changes compared to v2:
  rebased against next dated 11/05
  dropped changes to the patch class (should rework the class to align more closely with OE)
  make ci_build.sh --help output readable
  make the linux-custom recipe honor KERNEL_FILE
  configure qemu for mipsel

Changes compared to v1:
  the ".patch-commands" file should also be suffixed with "~"
  patch the hello package to test patch.bbclass from a dpkg recipe

Cedric Hombourger (6):
  wic-img: do not leave temporary files behind [SAME]
  isar-bootstrap: wipe out previously created rootfs with --one-file-system [SAME]
  base-apt-helper: fixed a few typos found in comments [SAME]
  ci_build.sh: delay "set -x" for a clean --help output [NEW]
  linux-custom: honor KERNEL_FILE setting [NEW]
  bitbake.conf: set name of the qemu-static binary to use for mipsel [NEW]

 meta/classes/base-apt-helper.bbclass                |  4 ++--
 meta/classes/image.bbclass                          |  1 -
 meta/classes/wic-img.bbclass                        |  8 ++++++--
 meta/conf/bitbake.conf                              |  2 ++
 meta/recipes-core/isar-bootstrap/isar-bootstrap.inc |  2 +-
 meta/recipes-kernel/linux/files/build-kernel.sh     | 10 +++++-----
 meta/recipes-kernel/linux/linux-custom.inc          |  1 +
 scripts/ci_build.sh                                 |  4 +++-
 8 files changed, 20 insertions(+), 12 deletions(-)

-- 
2.20.1

[PATCH v3 1/6] wic-img: do not leave temporary files behind

[Cedric Hombourger]

We do not seem to make any use of the timestamped wic images being
generated and worse we are leaving them in the buildchroot tmp folder.
"mv" the generated files instead of "cp"'ing them to the deploy
directory.

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/classes/wic-img.bbclass | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/meta/classes/wic-img.bbclass b/meta/classes/wic-img.bbclass
index 795640f..24c3b65 100644
--- a/meta/classes/wic-img.bbclass
+++ b/meta/classes/wic-img.bbclass
@@ -155,8 +155,12 @@ EOSUDO
             -e ${IMAGE_BASENAME} ${WIC_CREATE_EXTRA_ARGS}
     sudo chown -R $(stat -c "%U" ${ISARROOT}) ${ISARROOT}/meta ${ISARROOT}/meta-isar ${ISARROOT}/scripts || true
     WIC_DIRECT=$(ls -t -1 ${BUILDCHROOT_DIR}/$WICTMP/${IMAGE_FULLNAME}.wic/*.direct | head -1)
-    cp -f ${WIC_DIRECT} ${WIC_IMAGE_FILE}
-    cp -f ${WIC_DIRECT}.bmap ${WIC_IMAGE_FILE}.bmap
+    sudo chown $(stat -c "%U:%G" ${ISARROOT}) ${WIC_DIRECT} ${WIC_DIRECT}.bmap
+    sudo mv -f ${WIC_DIRECT} ${WIC_IMAGE_FILE}
+    sudo mv -f ${WIC_DIRECT}.bmap ${WIC_IMAGE_FILE}.bmap
+    sudo rmdir --ignore-fail-on-non-empty \
+        ${BUILDCHROOT_DIR}/${WICTMP}/${IMAGE_FULLNAME}.wic \
+        ${BUILDCHROOT_DIR}/${WICTMP}
 }
 
 do_wic_image[file-checksums] += "${WKS_FILE_CHECKSUM}"
-- 
2.20.1

[PATCH v3 2/6] isar-bootstrap: wipe out previously created rootfs with --one-file-system

[Cedric Hombourger]

As a safety precaution and to prevent the accidental deletion of files
from dangling bind mounts, remove the previously created rootfs with
rm -rf --one-file-system

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
index 1a58471..0fb033b 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -245,7 +245,7 @@ isar_bootstrap() {
     sudo -E -s <<'EOSUDO'
         set -e
         if [ ! -e "${DEPLOY_ISAR_BOOTSTRAP}" ]; then
-            rm -rf "${ROOTFSDIR}"
+            rm -rf --one-file-system "${ROOTFSDIR}"
 
             if [ ${IS_HOST} ]; then
                 ${DEBOOTSTRAP} $debootstrap_args \
-- 
2.20.1

[PATCH v3 3/6] base-apt-helper: fixed a few typos found in comments

[Cedric Hombourger]

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/classes/base-apt-helper.bbclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/base-apt-helper.bbclass b/meta/classes/base-apt-helper.bbclass
index 81df45a..31b2832 100644
--- a/meta/classes/base-apt-helper.bbclass
+++ b/meta/classes/base-apt-helper.bbclass
@@ -21,11 +21,11 @@ populate_base_apt() {
     find $search_dir -name '*.deb' | while read package; do
         # NOTE: due to packages stored by reprepro are not modified, we can
         # use search by filename to check if package is already in repo. In
-        # addition, m5sums could be compared to ensure, that package is the
+        # addition, md5sums are compared to ensure that the package is the
         # same and should not be overwritten. This method is easier and more
         # robust than querying reprepro by name.
 
-        # Check if this package is taken from Isar-apt, if so - ingore it.
+        # Check if this package is taken from Isar-apt, if so - ignore it.
         base_name=${package##*/}
         isar_package=$(find ${REPO_ISAR_DIR}/${DISTRO} -name $base_name)
         if [ -n "$isar_package" ]; then
-- 
2.20.1

[PATCH v3 4/6] ci_build.sh: delay "set -x" for a clean --help output

[Cedric Hombourger]

tracing (set -x) was enabled far too early and caused the output
of the --help to be barely readable. enable this after we have
parsed options.

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 scripts/ci_build.sh | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index ecaa88a..6efdfc0 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -4,7 +4,6 @@
 # Alexander Smirnov <asmirnov@ilbers.de>
 # Copyright (c) 2016-2018 ilbers GmbH
 
-set -x
 set -e
 
 ES_BUG=3
@@ -122,6 +121,9 @@ do
     shift
 done
 
+# the real stuff starts here, trace commands from now on
+set -x
+
 # Setup build folder for the current build
 if [ ! -d "$BUILD_DIR" ]; then
         mkdir -p "$BUILD_DIR"
-- 
2.20.1

[PATCH v3 5/6] linux-custom: honor KERNEL_FILE setting

[Cedric Hombourger]

vmlinuz was hard-coded in the postinst script generated by the
build-kernel.sh script. This really needs to be ${KERNEL_FILE}
to support architectures where a vmlinux file is used instead
(mipsel in particular). Rename the "vmlinuz" local variable to
kernel_file so that it is clear it is derived from KERNEL_FILE.

As we now have multiple recipes using KERNEL_FILE (image and
kernel recipes), its default setting (vmlinuz) was moved to
bitbake.conf

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/classes/image.bbclass                      |  1 -
 meta/conf/bitbake.conf                          |  1 +
 meta/recipes-kernel/linux/files/build-kernel.sh | 10 +++++-----
 meta/recipes-kernel/linux/linux-custom.inc      |  1 +
 4 files changed, 7 insertions(+), 6 deletions(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 8384b71..7690bc5 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -16,7 +16,6 @@ IMAGE_INSTALL += "${@ ("linux-image-" + d.getVar("KERNEL_NAME", True)) if d.getV
 IMAGE_FULLNAME = "${PN}-${DISTRO}-${MACHINE}"
 
 # These variables are used by wic and start_vm
-KERNEL_FILE  ?= "vmlinuz"
 KERNEL_IMAGE ?= "${IMAGE_FULLNAME}-${KERNEL_FILE}"
 INITRD_IMAGE ?= "${IMAGE_FULLNAME}-initrd.img"
 
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 1f20c98..d6329b3 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -56,6 +56,7 @@ BUILDCHROOT_HOST_DIR = "${DEPLOY_DIR_BUILDCHROOT}-host/${HOST_DISTRO}-${HOST_ARC
 BUILDCHROOT_TARGET_DIR = "${DEPLOY_DIR_BUILDCHROOT}-target/${DISTRO}-${DISTRO_ARCH}"
 SDKCHROOT_DIR = "${DEPLOY_DIR_SDKCHROOT}/${DISTRO}-${DISTRO_ARCH}"
 CACHE = "${TMPDIR}/cache"
+KERNEL_FILE ?= "vmlinuz"
 
 OVERRIDES = "${DISTRO_ARCH}:${MACHINE}:${DISTRO}:forcevariable"
 FILESOVERRIDES = "${DISTRO_ARCH}:${MACHINE}"
diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh
index 7b651af..b46cefa 100644
--- a/meta/recipes-kernel/linux/files/build-kernel.sh
+++ b/meta/recipes-kernel/linux/files/build-kernel.sh
@@ -90,7 +90,7 @@ if [ -f /lib/modules/${PV}/.fresh-install ]; then\\
 else\\
 	change=upgrade\\
 fi\\
-linux-update-symlinks \$change ${PV} /boot/vmlinuz-${PV}\\
+linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\
 rm -f /lib/modules/${PV}/.fresh-install"
 
 # Add Debian-like link removal to postrm
@@ -100,14 +100,14 @@ sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \
 rm -f /lib/modules/${PV}/.fresh-install\\
 \\
 if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks >/dev/null; then\\
-	linux-update-symlinks remove ${PV}  /boot/vmlinuz-${PV}\\
+	linux-update-symlinks remove ${PV}  /boot/${KERNEL_FILE}-${PV}\\
 fi"
 
 # Make sure arm64 kernels are decompressed
 if [ "$target_arch" = "arm64" ]; then
-	vmlinuz="${REPACK_LINUX_IMAGE_DIR}/boot/vmlinuz-${PV}"
-	mv "$vmlinuz" "$vmlinuz.gz"
-	gunzip "$vmlinuz.gz"
+	kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}"
+	mv "${kernel_file}" "${kernel_file}.gz"
+	gunzip "${kernel_file}.gz"
 fi
 
 dpkg-gencontrol -crepack/debian/control \
diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc
index dc2af1a..b597e25 100644
--- a/meta/recipes-kernel/linux/linux-custom.inc
+++ b/meta/recipes-kernel/linux/linux-custom.inc
@@ -72,6 +72,7 @@ dpkg_runbuild() {
 
 	export KBUILD_DEPENDS="${KBUILD_DEPENDS}"
 	export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}"
+	export KERNEL_FILE="${KERNEL_FILE}"
 	export KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}"
 
 	sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} ${PP}/build-kernel.sh ${PP}/${PPS} ${DISTRO_ARCH}
-- 
2.20.1

[PATCH v3 6/6] bitbake.conf: set name of the qemu-static binary to use for mipsel

[Cedric Hombourger]

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/conf/bitbake.conf | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index d6329b3..028abde 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -66,6 +66,7 @@ QEMU_ARCH_amd64 = "x86_64"
 QEMU_ARCH_i386 = "i386"
 QEMU_ARCH_armhf = "arm"
 QEMU_ARCH_arm64 = "aarch64"
+QEMU_ARCH_mipsel = "mipsel"
 
 # Codename of the repository created by the caching class
 DEBDISTRONAME = "isar"
-- 
2.20.1

[PATCH v4 - misc fixes 0/6] miscellaneous (minor) fixes

[Cedric Hombourger]

The following minor fixes are being submitted for review and inclusion to "next"

Changes compared to v3:
  change default KERNEL_FILE setting for mipsel

Changes compared to v2:
  rebased against next dated 11/05
  dropped changes to the patch class (should rework the class to align more closely with OE)
  make ci_build.sh --help output readable
  make the linux-custom recipe honor KERNEL_FILE
  configure qemu for mipsel

Cedric Hombourger (6):
  wic-img: do not leave temporary files behind [SAME]
  isar-bootstrap: wipe out previously created rootfs with --one-file-system [SAME]
  base-apt-helper: fixed a few typos found in comments [SAME]
  ci_build.sh: delay "set -x" for a clean --help output [SAME]
  linux-custom: honor KERNEL_FILE setting [MODIFIED]
  bitbake.conf: set name of the qemu-static binary to use for mipsel [SAME]

 meta-isar/conf/machine/qemumipsel.conf              |  2 --
 meta/classes/base-apt-helper.bbclass                |  4 ++--
 meta/classes/image.bbclass                          |  1 -
 meta/classes/wic-img.bbclass                        |  8 ++++++--
 meta/conf/bitbake.conf                              |  3 +++
 meta/recipes-core/isar-bootstrap/isar-bootstrap.inc |  2 +-
 meta/recipes-kernel/linux/files/build-kernel.sh     | 10 +++++-----
 meta/recipes-kernel/linux/linux-custom.inc          |  1 +
 scripts/ci_build.sh                                 |  4 +++-
 9 files changed, 21 insertions(+), 14 deletions(-)

-- 
2.20.1

[PATCH v4 - misc fixes 1/6] wic-img: do not leave temporary files behind

[Cedric Hombourger]

We do not seem to make any use of the timestamped wic images being
generated and worse we are leaving them in the buildchroot tmp folder.
"mv" the generated files instead of "cp"'ing them to the deploy
directory.

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/classes/wic-img.bbclass | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/meta/classes/wic-img.bbclass b/meta/classes/wic-img.bbclass
index 795640f..24c3b65 100644
--- a/meta/classes/wic-img.bbclass
+++ b/meta/classes/wic-img.bbclass
@@ -155,8 +155,12 @@ EOSUDO
             -e ${IMAGE_BASENAME} ${WIC_CREATE_EXTRA_ARGS}
     sudo chown -R $(stat -c "%U" ${ISARROOT}) ${ISARROOT}/meta ${ISARROOT}/meta-isar ${ISARROOT}/scripts || true
     WIC_DIRECT=$(ls -t -1 ${BUILDCHROOT_DIR}/$WICTMP/${IMAGE_FULLNAME}.wic/*.direct | head -1)
-    cp -f ${WIC_DIRECT} ${WIC_IMAGE_FILE}
-    cp -f ${WIC_DIRECT}.bmap ${WIC_IMAGE_FILE}.bmap
+    sudo chown $(stat -c "%U:%G" ${ISARROOT}) ${WIC_DIRECT} ${WIC_DIRECT}.bmap
+    sudo mv -f ${WIC_DIRECT} ${WIC_IMAGE_FILE}
+    sudo mv -f ${WIC_DIRECT}.bmap ${WIC_IMAGE_FILE}.bmap
+    sudo rmdir --ignore-fail-on-non-empty \
+        ${BUILDCHROOT_DIR}/${WICTMP}/${IMAGE_FULLNAME}.wic \
+        ${BUILDCHROOT_DIR}/${WICTMP}
 }
 
 do_wic_image[file-checksums] += "${WKS_FILE_CHECKSUM}"
-- 
2.20.1

Re: [PATCH v4 - misc fixes 1/6] wic-img: do not leave temporary files behind

[Jan Kiszka]

On 08.11.19 10:23, Cedric Hombourger wrote:
> We do not seem to make any use of the timestamped wic images being
> generated and worse we are leaving them in the buildchroot tmp folder.
> "mv" the generated files instead of "cp"'ing them to the deploy
> directory.
> 
> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
> ---
>   meta/classes/wic-img.bbclass | 8 ++++++--
>   1 file changed, 6 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/classes/wic-img.bbclass b/meta/classes/wic-img.bbclass
> index 795640f..24c3b65 100644
> --- a/meta/classes/wic-img.bbclass
> +++ b/meta/classes/wic-img.bbclass
> @@ -155,8 +155,12 @@ EOSUDO
>               -e ${IMAGE_BASENAME} ${WIC_CREATE_EXTRA_ARGS}
>       sudo chown -R $(stat -c "%U" ${ISARROOT}) ${ISARROOT}/meta ${ISARROOT}/meta-isar ${ISARROOT}/scripts || true
>       WIC_DIRECT=$(ls -t -1 ${BUILDCHROOT_DIR}/$WICTMP/${IMAGE_FULLNAME}.wic/*.direct | head -1)
> -    cp -f ${WIC_DIRECT} ${WIC_IMAGE_FILE}
> -    cp -f ${WIC_DIRECT}.bmap ${WIC_IMAGE_FILE}.bmap
> +    sudo chown $(stat -c "%U:%G" ${ISARROOT}) ${WIC_DIRECT} ${WIC_DIRECT}.bmap
> +    sudo mv -f ${WIC_DIRECT} ${WIC_IMAGE_FILE}
> +    sudo mv -f ${WIC_DIRECT}.bmap ${WIC_IMAGE_FILE}.bmap
> +    sudo rmdir --ignore-fail-on-non-empty \
> +        ${BUILDCHROOT_DIR}/${WICTMP}/${IMAGE_FULLNAME}.wic \
> +        ${BUILDCHROOT_DIR}/${WICTMP}

Where does the need for the sudos come from? What are source and target 
users here?

Jan

>   }
>   
>   do_wic_image[file-checksums] += "${WKS_FILE_CHECKSUM}"
> 

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH v4 - misc fixes 1/6] wic-img: do not leave temporary files behind

[Cedric Hombourger]

On 11/15/19 2:38 PM, Jan Kiszka wrote:
> On 08.11.19 10:23, Cedric Hombourger wrote:
>> We do not seem to make any use of the timestamped wic images being
>> generated and worse we are leaving them in the buildchroot tmp folder.
>> "mv" the generated files instead of "cp"'ing them to the deploy
>> directory.
>>
>> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
>> ---
>>   meta/classes/wic-img.bbclass | 8 ++++++--
>>   1 file changed, 6 insertions(+), 2 deletions(-)
>>
>> diff --git a/meta/classes/wic-img.bbclass b/meta/classes/wic-img.bbclass
>> index 795640f..24c3b65 100644
>> --- a/meta/classes/wic-img.bbclass
>> +++ b/meta/classes/wic-img.bbclass
>> @@ -155,8 +155,12 @@ EOSUDO
>>               -e ${IMAGE_BASENAME} ${WIC_CREATE_EXTRA_ARGS}
>>       sudo chown -R $(stat -c "%U" ${ISARROOT}) ${ISARROOT}/meta 
>> ${ISARROOT}/meta-isar ${ISARROOT}/scripts || true
>>       WIC_DIRECT=$(ls -t -1 
>> ${BUILDCHROOT_DIR}/$WICTMP/${IMAGE_FULLNAME}.wic/*.direct | head -1)
>> -    cp -f ${WIC_DIRECT} ${WIC_IMAGE_FILE}
>> -    cp -f ${WIC_DIRECT}.bmap ${WIC_IMAGE_FILE}.bmap
>> +    sudo chown $(stat -c "%U:%G" ${ISARROOT}) ${WIC_DIRECT} 
>> ${WIC_DIRECT}.bmap
>> +    sudo mv -f ${WIC_DIRECT} ${WIC_IMAGE_FILE}
>> +    sudo mv -f ${WIC_DIRECT}.bmap ${WIC_IMAGE_FILE}.bmap
>> +    sudo rmdir --ignore-fail-on-non-empty \
>> +        ${BUILDCHROOT_DIR}/${WICTMP}/${IMAGE_FULLNAME}.wic \
>> +        ${BUILDCHROOT_DIR}/${WICTMP}
>
> Where does the need for the sudos come from? What are source and 
> target users here?
>
files created by wic are owned by root. I am happy to check if the 
following sudos are truly required
(sudo wasn't required before because the files were copied and not 
moved, so ro access was sufficient)

> Jan
>
>>   }
>>     do_wic_image[file-checksums] += "${WKS_FILE_CHECKSUM}"
>>
>

[PATCH v5 0/6] miscellaneous (minor) fixes

[Cedric Hombourger]

The following minor fixes are being submitted for review and inclusion to "next"

Changes compared to v4:
  kill a few sudos when moving wic files (chown WICTMP after running wic so we can
  "mv" generated files to DEPLOY_DIR (they are otherwise owned by root:root))

Changes compared to v3:
  change default KERNEL_FILE setting for mipsel

Changes compared to v2:
  rebased against next dated 11/05
  dropped changes to the patch class (should rework the class to align more closely with OE)
  make ci_build.sh --help output readable
  make the linux-custom recipe honor KERNEL_FILE
  configure qemu for mipsel

Cedric Hombourger (6):
  wic-img: do not leave temporary files behind [CHANGED]
  isar-bootstrap: wipe out previously created rootfs with --one-file-system [SAME]
  base-apt-helper: fixed a few typos found in comments [SAME]
  ci_build.sh: delay "set -x" for a clean --help output [SAME]
  linux-custom: honor KERNEL_FILE setting [SAME]
  bitbake.conf: set name of the qemu-static binary to use for mipsel [SAME]

 meta-isar/conf/machine/qemumipsel.conf              |  2 --
 meta/classes/base-apt-helper.bbclass                |  4 ++--
 meta/classes/image.bbclass                          |  1 -
 meta/classes/wic-img.bbclass                        |  6 ++++--
 meta/conf/bitbake.conf                              |  3 +++
 meta/recipes-core/isar-bootstrap/isar-bootstrap.inc |  2 +-
 meta/recipes-kernel/linux/files/build-kernel.sh     | 10 +++++-----
 meta/recipes-kernel/linux/linux-custom.inc          |  1 +
 scripts/ci_build.sh                                 |  4 +++-
 9 files changed, 19 insertions(+), 14 deletions(-)

-- 
2.20.1

[PATCH v5 1/6] wic-img: do not leave temporary files behind

[Cedric Hombourger]

We do not seem to make any use of the timestamped wic images being
generated and worse we are leaving them in the buildchroot tmp folder.
"mv" the generated files instead of "cp"'ing them to the deploy
directory.

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/classes/wic-img.bbclass | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/meta/classes/wic-img.bbclass b/meta/classes/wic-img.bbclass
index 795640f..7d30e45 100644
--- a/meta/classes/wic-img.bbclass
+++ b/meta/classes/wic-img.bbclass
@@ -155,8 +155,10 @@ EOSUDO
             -e ${IMAGE_BASENAME} ${WIC_CREATE_EXTRA_ARGS}
     sudo chown -R $(stat -c "%U" ${ISARROOT}) ${ISARROOT}/meta ${ISARROOT}/meta-isar ${ISARROOT}/scripts || true
     WIC_DIRECT=$(ls -t -1 ${BUILDCHROOT_DIR}/$WICTMP/${IMAGE_FULLNAME}.wic/*.direct | head -1)
-    cp -f ${WIC_DIRECT} ${WIC_IMAGE_FILE}
-    cp -f ${WIC_DIRECT}.bmap ${WIC_IMAGE_FILE}.bmap
+    sudo chown -R $(id -u):$(id -g) ${BUILDCHROOT_DIR}/${WICTMP}
+    mv -f ${WIC_DIRECT} ${WIC_IMAGE_FILE}
+    mv -f ${WIC_DIRECT}.bmap ${WIC_IMAGE_FILE}.bmap
+    rm -rf ${BUILDCHROOT_DIR}/${WICTMP}
 }
 
 do_wic_image[file-checksums] += "${WKS_FILE_CHECKSUM}"
-- 
2.20.1

[PATCH v5 2/6] isar-bootstrap: wipe out previously created rootfs with --one-file-system

[Cedric Hombourger]

As a safety precaution and to prevent the accidental deletion of files
from dangling bind mounts, remove the previously created rootfs with
rm -rf --one-file-system

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
index b506f30..271892c 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -241,7 +241,7 @@ isar_bootstrap() {
     sudo -E -s <<'EOSUDO'
         set -e
         if [ ! -e "${DEPLOY_ISAR_BOOTSTRAP}" ]; then
-            rm -rf "${ROOTFSDIR}"
+            rm -rf --one-file-system "${ROOTFSDIR}"
 
             if [ ${IS_HOST} ]; then
                 ${DEBOOTSTRAP} $debootstrap_args \
-- 
2.20.1

[PATCH v5 3/6] base-apt-helper: fixed a few typos found in comments

[Cedric Hombourger]

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/classes/base-apt-helper.bbclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/base-apt-helper.bbclass b/meta/classes/base-apt-helper.bbclass
index d5674c9..90b2cfc 100644
--- a/meta/classes/base-apt-helper.bbclass
+++ b/meta/classes/base-apt-helper.bbclass
@@ -21,11 +21,11 @@ populate_base_apt() {
     find $search_dir -name '*.deb' | while read package; do
         # NOTE: due to packages stored by reprepro are not modified, we can
         # use search by filename to check if package is already in repo. In
-        # addition, m5sums could be compared to ensure, that package is the
+        # addition, md5sums are compared to ensure that the package is the
         # same and should not be overwritten. This method is easier and more
         # robust than querying reprepro by name.
 
-        # Check if this package is taken from Isar-apt, if so - ingore it.
+        # Check if this package is taken from Isar-apt, if so - ignore it.
         base_name=${package##*/}
         isar_apt_p=$(find ${REPO_ISAR_DIR}/${DISTRO} -name $base_name)
         if [ -n "$isar_apt_p" ]; then
-- 
2.20.1

[PATCH v5 4/6] ci_build.sh: delay "set -x" for a clean --help output

[Cedric Hombourger]

tracing (set -x) was enabled far too early and caused the output
of the --help to be barely readable. enable this after we have
parsed options.

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 scripts/ci_build.sh | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index ecaa88a..6efdfc0 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -4,7 +4,6 @@
 # Alexander Smirnov <asmirnov@ilbers.de>
 # Copyright (c) 2016-2018 ilbers GmbH
 
-set -x
 set -e
 
 ES_BUG=3
@@ -122,6 +121,9 @@ do
     shift
 done
 
+# the real stuff starts here, trace commands from now on
+set -x
+
 # Setup build folder for the current build
 if [ ! -d "$BUILD_DIR" ]; then
         mkdir -p "$BUILD_DIR"
-- 
2.20.1

[PATCH v5 5/6] linux-custom: honor KERNEL_FILE setting

[Cedric Hombourger]

vmlinuz was hard-coded in the postinst script generated by the
build-kernel.sh script. This really needs to be ${KERNEL_FILE}
to support architectures where a vmlinux file is used instead
(mipsel in particular). Rename the "vmlinuz" local variable to
kernel_file so that it is clear it is derived from KERNEL_FILE.

As we now have multiple recipes using KERNEL_FILE (image and
kernel recipes), its default setting (vmlinuz) was moved to
bitbake.conf (defaults to vmlinuz for all architectures but
mipsel where it is set to vmlinux)

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta-isar/conf/machine/qemumipsel.conf          |  2 --
 meta/classes/image.bbclass                      |  1 -
 meta/conf/bitbake.conf                          |  2 ++
 meta/recipes-kernel/linux/files/build-kernel.sh | 10 +++++-----
 meta/recipes-kernel/linux/linux-custom.inc      |  1 +
 5 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/meta-isar/conf/machine/qemumipsel.conf b/meta-isar/conf/machine/qemumipsel.conf
index 6ca0777..18191fc 100644
--- a/meta-isar/conf/machine/qemumipsel.conf
+++ b/meta-isar/conf/machine/qemumipsel.conf
@@ -1,8 +1,6 @@
 # This software is a part of ISAR.
 # Copyright (C) 2019 Mentor Graphics, a Siemens business
 
-KERNEL_FILE ?= "vmlinux"
-
 MACHINE_SERIAL ?= "ttyS0"
 BAUDRATE_TTY ?= "115200"
 
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 8384b71..7690bc5 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -16,7 +16,6 @@ IMAGE_INSTALL += "${@ ("linux-image-" + d.getVar("KERNEL_NAME", True)) if d.getV
 IMAGE_FULLNAME = "${PN}-${DISTRO}-${MACHINE}"
 
 # These variables are used by wic and start_vm
-KERNEL_FILE  ?= "vmlinuz"
 KERNEL_IMAGE ?= "${IMAGE_FULLNAME}-${KERNEL_FILE}"
 INITRD_IMAGE ?= "${IMAGE_FULLNAME}-initrd.img"
 
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 1f20c98..e45284c 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -56,6 +56,8 @@ BUILDCHROOT_HOST_DIR = "${DEPLOY_DIR_BUILDCHROOT}-host/${HOST_DISTRO}-${HOST_ARC
 BUILDCHROOT_TARGET_DIR = "${DEPLOY_DIR_BUILDCHROOT}-target/${DISTRO}-${DISTRO_ARCH}"
 SDKCHROOT_DIR = "${DEPLOY_DIR_SDKCHROOT}/${DISTRO}-${DISTRO_ARCH}"
 CACHE = "${TMPDIR}/cache"
+KERNEL_FILE ?= "vmlinuz"
+KERNEL_FILE_mipsel ?= "vmlinux"
 
 OVERRIDES = "${DISTRO_ARCH}:${MACHINE}:${DISTRO}:forcevariable"
 FILESOVERRIDES = "${DISTRO_ARCH}:${MACHINE}"
diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh
index 7b651af..b46cefa 100644
--- a/meta/recipes-kernel/linux/files/build-kernel.sh
+++ b/meta/recipes-kernel/linux/files/build-kernel.sh
@@ -90,7 +90,7 @@ if [ -f /lib/modules/${PV}/.fresh-install ]; then\\
 else\\
 	change=upgrade\\
 fi\\
-linux-update-symlinks \$change ${PV} /boot/vmlinuz-${PV}\\
+linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\
 rm -f /lib/modules/${PV}/.fresh-install"
 
 # Add Debian-like link removal to postrm
@@ -100,14 +100,14 @@ sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \
 rm -f /lib/modules/${PV}/.fresh-install\\
 \\
 if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks >/dev/null; then\\
-	linux-update-symlinks remove ${PV}  /boot/vmlinuz-${PV}\\
+	linux-update-symlinks remove ${PV}  /boot/${KERNEL_FILE}-${PV}\\
 fi"
 
 # Make sure arm64 kernels are decompressed
 if [ "$target_arch" = "arm64" ]; then
-	vmlinuz="${REPACK_LINUX_IMAGE_DIR}/boot/vmlinuz-${PV}"
-	mv "$vmlinuz" "$vmlinuz.gz"
-	gunzip "$vmlinuz.gz"
+	kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}"
+	mv "${kernel_file}" "${kernel_file}.gz"
+	gunzip "${kernel_file}.gz"
 fi
 
 dpkg-gencontrol -crepack/debian/control \
diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc
index fcbe326..17f8464 100644
--- a/meta/recipes-kernel/linux/linux-custom.inc
+++ b/meta/recipes-kernel/linux/linux-custom.inc
@@ -72,6 +72,7 @@ dpkg_runbuild() {
 
 	export KBUILD_DEPENDS="${KBUILD_DEPENDS}"
 	export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}"
+	export KERNEL_FILE="${KERNEL_FILE}"
 	export KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}"
 
 	sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} ${PP}/build-kernel.sh ${PP}/${PPS} ${DISTRO_ARCH}
-- 
2.20.1

[PATCH v5 6/6] bitbake.conf: set name of the qemu-static binary to use for mipsel

[Cedric Hombourger]

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/conf/bitbake.conf | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index e45284c..0eb3b54 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -67,6 +67,7 @@ QEMU_ARCH_amd64 = "x86_64"
 QEMU_ARCH_i386 = "i386"
 QEMU_ARCH_armhf = "arm"
 QEMU_ARCH_arm64 = "aarch64"
+QEMU_ARCH_mipsel = "mipsel"
 
 # Codename of the repository created by the caching class
 DEBDISTRONAME = "isar"
-- 
2.20.1

Re: [PATCH v5 0/6] miscellaneous (minor) fixes

[Baurzhan Ismagulov]

On Fri, Nov 15, 2019 at 06:00:09PM +0100, Cedric Hombourger wrote:
> The following minor fixes are being submitted for review and inclusion to "next"
> 
> Changes compared to v4:
>   kill a few sudos when moving wic files (chown WICTMP after running wic so we can
>   "mv" generated files to DEPLOY_DIR (they are otherwise owned by root:root))

Thanks, applied to next.

With kind regards,
Baurzhan.

[PATCH v4 - misc fixes 2/6] isar-bootstrap: wipe out previously created rootfs with --one-file-system

[Cedric Hombourger]

As a safety precaution and to prevent the accidental deletion of files
from dangling bind mounts, remove the previously created rootfs with
rm -rf --one-file-system

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/recipes-core/isar-bootstrap/isar-bootstrap.inc | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
index 1a58471..0fb033b 100644
--- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
+++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
@@ -245,7 +245,7 @@ isar_bootstrap() {
     sudo -E -s <<'EOSUDO'
         set -e
         if [ ! -e "${DEPLOY_ISAR_BOOTSTRAP}" ]; then
-            rm -rf "${ROOTFSDIR}"
+            rm -rf --one-file-system "${ROOTFSDIR}"
 
             if [ ${IS_HOST} ]; then
                 ${DEBOOTSTRAP} $debootstrap_args \
-- 
2.20.1

[PATCH v4 - misc fixes 3/6] base-apt-helper: fixed a few typos found in comments

[Cedric Hombourger]

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/classes/base-apt-helper.bbclass | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/base-apt-helper.bbclass b/meta/classes/base-apt-helper.bbclass
index 81df45a..31b2832 100644
--- a/meta/classes/base-apt-helper.bbclass
+++ b/meta/classes/base-apt-helper.bbclass
@@ -21,11 +21,11 @@ populate_base_apt() {
     find $search_dir -name '*.deb' | while read package; do
         # NOTE: due to packages stored by reprepro are not modified, we can
         # use search by filename to check if package is already in repo. In
-        # addition, m5sums could be compared to ensure, that package is the
+        # addition, md5sums are compared to ensure that the package is the
         # same and should not be overwritten. This method is easier and more
         # robust than querying reprepro by name.
 
-        # Check if this package is taken from Isar-apt, if so - ingore it.
+        # Check if this package is taken from Isar-apt, if so - ignore it.
         base_name=${package##*/}
         isar_package=$(find ${REPO_ISAR_DIR}/${DISTRO} -name $base_name)
         if [ -n "$isar_package" ]; then
-- 
2.20.1

[PATCH v4 - misc fixes 4/6] ci_build.sh: delay "set -x" for a clean --help output

[Cedric Hombourger]

tracing (set -x) was enabled far too early and caused the output
of the --help to be barely readable. enable this after we have
parsed options.

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 scripts/ci_build.sh | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index ecaa88a..6efdfc0 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -4,7 +4,6 @@
 # Alexander Smirnov <asmirnov@ilbers.de>
 # Copyright (c) 2016-2018 ilbers GmbH
 
-set -x
 set -e
 
 ES_BUG=3
@@ -122,6 +121,9 @@ do
     shift
 done
 
+# the real stuff starts here, trace commands from now on
+set -x
+
 # Setup build folder for the current build
 if [ ! -d "$BUILD_DIR" ]; then
         mkdir -p "$BUILD_DIR"
-- 
2.20.1

[PATCH v4 - misc fixes 5/6] linux-custom: honor KERNEL_FILE setting

[Cedric Hombourger]

vmlinuz was hard-coded in the postinst script generated by the
build-kernel.sh script. This really needs to be ${KERNEL_FILE}
to support architectures where a vmlinux file is used instead
(mipsel in particular). Rename the "vmlinuz" local variable to
kernel_file so that it is clear it is derived from KERNEL_FILE.

As we now have multiple recipes using KERNEL_FILE (image and
kernel recipes), its default setting (vmlinuz) was moved to
bitbake.conf (defaults to vmlinuz for all architectures but
mipsel where it is set to vmlinux)

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta-isar/conf/machine/qemumipsel.conf          |  2 --
 meta/classes/image.bbclass                      |  1 -
 meta/conf/bitbake.conf                          |  2 ++
 meta/recipes-kernel/linux/files/build-kernel.sh | 10 +++++-----
 meta/recipes-kernel/linux/linux-custom.inc      |  1 +
 5 files changed, 8 insertions(+), 8 deletions(-)

diff --git a/meta-isar/conf/machine/qemumipsel.conf b/meta-isar/conf/machine/qemumipsel.conf
index 6ca0777..18191fc 100644
--- a/meta-isar/conf/machine/qemumipsel.conf
+++ b/meta-isar/conf/machine/qemumipsel.conf
@@ -1,8 +1,6 @@
 # This software is a part of ISAR.
 # Copyright (C) 2019 Mentor Graphics, a Siemens business
 
-KERNEL_FILE ?= "vmlinux"
-
 MACHINE_SERIAL ?= "ttyS0"
 BAUDRATE_TTY ?= "115200"
 
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 8384b71..7690bc5 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -16,7 +16,6 @@ IMAGE_INSTALL += "${@ ("linux-image-" + d.getVar("KERNEL_NAME", True)) if d.getV
 IMAGE_FULLNAME = "${PN}-${DISTRO}-${MACHINE}"
 
 # These variables are used by wic and start_vm
-KERNEL_FILE  ?= "vmlinuz"
 KERNEL_IMAGE ?= "${IMAGE_FULLNAME}-${KERNEL_FILE}"
 INITRD_IMAGE ?= "${IMAGE_FULLNAME}-initrd.img"
 
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 1f20c98..e45284c 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -56,6 +56,8 @@ BUILDCHROOT_HOST_DIR = "${DEPLOY_DIR_BUILDCHROOT}-host/${HOST_DISTRO}-${HOST_ARC
 BUILDCHROOT_TARGET_DIR = "${DEPLOY_DIR_BUILDCHROOT}-target/${DISTRO}-${DISTRO_ARCH}"
 SDKCHROOT_DIR = "${DEPLOY_DIR_SDKCHROOT}/${DISTRO}-${DISTRO_ARCH}"
 CACHE = "${TMPDIR}/cache"
+KERNEL_FILE ?= "vmlinuz"
+KERNEL_FILE_mipsel ?= "vmlinux"
 
 OVERRIDES = "${DISTRO_ARCH}:${MACHINE}:${DISTRO}:forcevariable"
 FILESOVERRIDES = "${DISTRO_ARCH}:${MACHINE}"
diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh
index 7b651af..b46cefa 100644
--- a/meta/recipes-kernel/linux/files/build-kernel.sh
+++ b/meta/recipes-kernel/linux/files/build-kernel.sh
@@ -90,7 +90,7 @@ if [ -f /lib/modules/${PV}/.fresh-install ]; then\\
 else\\
 	change=upgrade\\
 fi\\
-linux-update-symlinks \$change ${PV} /boot/vmlinuz-${PV}\\
+linux-update-symlinks \$change ${PV} /boot/${KERNEL_FILE}-${PV}\\
 rm -f /lib/modules/${PV}/.fresh-install"
 
 # Add Debian-like link removal to postrm
@@ -100,14 +100,14 @@ sed -i "${REPACK_LINUX_IMAGE_DIR}/DEBIAN/postrm" \
 rm -f /lib/modules/${PV}/.fresh-install\\
 \\
 if [ \"\$1\" != upgrade ] && command -v linux-update-symlinks >/dev/null; then\\
-	linux-update-symlinks remove ${PV}  /boot/vmlinuz-${PV}\\
+	linux-update-symlinks remove ${PV}  /boot/${KERNEL_FILE}-${PV}\\
 fi"
 
 # Make sure arm64 kernels are decompressed
 if [ "$target_arch" = "arm64" ]; then
-	vmlinuz="${REPACK_LINUX_IMAGE_DIR}/boot/vmlinuz-${PV}"
-	mv "$vmlinuz" "$vmlinuz.gz"
-	gunzip "$vmlinuz.gz"
+	kernel_file="${REPACK_LINUX_IMAGE_DIR}/boot/${KERNEL_FILE}-${PV}"
+	mv "${kernel_file}" "${kernel_file}.gz"
+	gunzip "${kernel_file}.gz"
 fi
 
 dpkg-gencontrol -crepack/debian/control \
diff --git a/meta/recipes-kernel/linux/linux-custom.inc b/meta/recipes-kernel/linux/linux-custom.inc
index dc2af1a..b597e25 100644
--- a/meta/recipes-kernel/linux/linux-custom.inc
+++ b/meta/recipes-kernel/linux/linux-custom.inc
@@ -72,6 +72,7 @@ dpkg_runbuild() {
 
 	export KBUILD_DEPENDS="${KBUILD_DEPENDS}"
 	export KERNEL_DEBIAN_DEPENDS="${KERNEL_DEBIAN_DEPENDS}"
+	export KERNEL_FILE="${KERNEL_FILE}"
 	export KERNEL_HEADERS_DEBIAN_DEPENDS="${KERNEL_HEADERS_DEBIAN_DEPENDS}"
 
 	sudo -E chroot --userspec=$( id -u ):$( id -g ) ${BUILDCHROOT_DIR} ${PP}/build-kernel.sh ${PP}/${PPS} ${DISTRO_ARCH}
-- 
2.20.1

[PATCH v4 - misc fixes 6/6] bitbake.conf: set name of the qemu-static binary to use for mipsel

[Cedric Hombourger]

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/conf/bitbake.conf | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index e45284c..0eb3b54 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -67,6 +67,7 @@ QEMU_ARCH_amd64 = "x86_64"
 QEMU_ARCH_i386 = "i386"
 QEMU_ARCH_armhf = "arm"
 QEMU_ARCH_arm64 = "aarch64"
+QEMU_ARCH_mipsel = "mipsel"
 
 # Codename of the repository created by the caching class
 DEBDISTRONAME = "isar"
-- 
2.20.1

[PATCH v4 - misc fixes 0/6] miscellaneous (minor) fixes

[chombourger]

[-- Attachment #1: Type: text/plain, Size: 88 bytes --]

Hello all,

Is this patch series ready for merge or is there anything that needs fixing?

Re: [PATCH v4 - misc fixes 0/6] miscellaneous (minor) fixes

[Baurzhan Ismagulov]

Hello Cedric,

On Tue, Nov 12, 2019 at 11:20:15AM -0800, chombourger@gmail.com wrote:
> Is this patch series ready for merge or is there anything that needs fixing?

Looks good to me. I've applied it to ibr/next. I'll do some more checks and let
you know (the commid id will change). Please check #3, it failed for some
reason, I had to apply it manually.

Regarding #1, it's an upstream feature. Do you know how Yocto deals with that?
E.g., does clean remove all of them? I found it useful once, when something was
broken and I could check the previous image. Also, I usually prefer not to
remove intermediate files to be able to debug stuff after the build. In this
case I agree that it's better to remove them.

Regarding #5, this is what many new users complain about: The configuration is
scattered through many files, it's difficult even to identify which variables
are available. MACHINE-centric tools with a single config file are much easier
to learn in this regard. Documenting stuff would be one option. But providing
few self-explaining locations with all variables is also important. Maybe we
could introduce meta/conf/arch/machine hierarchy and provide variables like
KERNEL_FILE there, at the cost of some repetition.

With kind regards,
Baurzhan.

Re: [PATCH v4 - misc fixes 0/6] miscellaneous (minor) fixes

[chombourger]

[-- Attachment #1.1: Type: text/plain, Size: 2300 bytes --]



On Friday, November 15, 2019 at 2:46:53 PM UTC+1, Baurzhan Ismagulov wrote:
>
> Hello Cedric, 
>
> Hello!
 

> On Tue, Nov 12, 2019 at 11:20:15AM -0800, chomb...@gmail.com <javascript:> 
> wrote: 
> > Is this patch series ready for merge or is there anything that needs 
> fixing? 
>
> Looks good to me. I've applied it to ibr/next. I'll do some more checks 
> and let 
> you know (the commid id will change). Please check #3, it failed for some 
> reason, I had to apply it manually. 
>

Rebased locally (it did apply cleanly for me) - will anyway send v5
 

>
> Regarding #1, it's an upstream feature. Do you know how Yocto deals with 
> that? 
> E.g., does clean remove all of them? I found it useful once, when 
> something was 
> broken and I could check the previous image. Also, I usually prefer not to 
> remove intermediate files to be able to debug stuff after the build. In 
> this 
> case I agree that it's better to remove them. 
>

I can check what upstream does but to answer the other question, clean 
unfortunately leaves them behind
WICTMP is created with mktemp -d so I wonder if we would have to store it 
somewhere if we wanted to keep files in the buildchroot
but delete them when we run do_clean
 
should we introduce WIC_DEBUG?= "0" and keep them if it was set to "1"?


> Regarding #5, this is what many new users complain about: The 
> configuration is 
> scattered through many files, it's difficult even to identify which 
> variables 
> are available. MACHINE-centric tools with a single config file are much 
> easier 
> to learn in this regard. Documenting stuff would be one option. But 
> providing 
> few self-explaining locations with all variables is also important. Maybe 
> we 
> could introduce meta/conf/arch/machine hierarchy and provide variables 
> like 
> KERNEL_FILE there, at the cost of some repetition. 
>

that sounds like a good idea but do we want to expand the scope of this 
patch series?
It would be nice to fix mips builds with our current design. 

As an aside, do we have a central TODO list where we can record things we 
really want to
get done before we release something like 1.0? I recall seeing e-mails from 
different people
with own lists. Should we have it in the repo so it's easy to find?
  

> With kind regards, 
> Baurzhan. 
>

[-- Attachment #1.2: Type: text/html, Size: 3439 bytes --]

Re: [PATCH v4 - misc fixes 0/6] miscellaneous (minor) fixes

[Baurzhan Ismagulov]

On Fri, Nov 15, 2019 at 08:59:15AM -0800, chombourger@gmail.com wrote:
> Rebased locally (it did apply cleanly for me) - will anyway send v5

Yes, I've seen this behavior -- rebase is fine, git am fails. Usually that was
due the changes in the context lines (git am is possibly pickier than rebase
for whatever reason), but in this case I couldn't identify any changes in the
context lines, either.


> I can check what upstream does but to answer the other question, clean 
> unfortunately leaves them behind

Isar -- yes, I had checked. The question was about the Yocto clean. Just in
case anyone has an answer.


> WICTMP is created with mktemp -d so I wonder if we would have to store it 
> somewhere if we wanted to keep files in the buildchroot
> but delete them when we run do_clean
>  
> should we introduce WIC_DEBUG?= "0" and keep them if it was set to "1"?

To make it clear, overall I'm willing to apply the series as is in v4. I'll
check v5 and let you know. I had thought about WIC_DEBUG, but I think it isn't
worth the effort. We don't modify the file, so having copies is useful only for
rare cases. OTOH, it eats space and the user might not be aware of that.


> that sounds like a good idea but do we want to expand the scope of this 
> patch series?

No, I'd also like to avoid expanding the scope.


> As an aside, do we have a central TODO list where we can record things we 
> really want to
> get done before we release something like 1.0? I recall seeing e-mails from 
> different people
> with own lists. Should we have it in the repo so it's easy to find?

We used to have it. We've removed it after a discussion here and possibly
offline. Some issues have been moved to GitHub. Feel free to create new issues
there.


With kind regards,
Baurzhan.

Re: [PATCH v4 - misc fixes 0/6] miscellaneous (minor) fixes

[chombourger]

[-- Attachment #1.1: Type: text/plain, Size: 1705 bytes --]



On Friday, November 8, 2019 at 10:23:36 AM UTC+1, Cedric Hombourger wrote:
>
> The following minor fixes are being submitted for review and inclusion to 
> "next" 
>
>
any feedback against v4 of this patch series?
 

> Changes compared to v3: 
>   change default KERNEL_FILE setting for mipsel 
>
> Changes compared to v2: 
>   rebased against next dated 11/05 
>   dropped changes to the patch class (should rework the class to align 
> more closely with OE) 
>   make ci_build.sh --help output readable 
>   make the linux-custom recipe honor KERNEL_FILE 
>   configure qemu for mipsel 
>
> Cedric Hombourger (6): 
>   wic-img: do not leave temporary files behind [SAME] 
>   isar-bootstrap: wipe out previously created rootfs with 
> --one-file-system [SAME] 
>   base-apt-helper: fixed a few typos found in comments [SAME] 
>   ci_build.sh: delay "set -x" for a clean --help output [SAME] 
>   linux-custom: honor KERNEL_FILE setting [MODIFIED] 
>   bitbake.conf: set name of the qemu-static binary to use for mipsel 
> [SAME] 
>
>  meta-isar/conf/machine/qemumipsel.conf              |  2 -- 
>  meta/classes/base-apt-helper.bbclass                |  4 ++-- 
>  meta/classes/image.bbclass                          |  1 - 
>  meta/classes/wic-img.bbclass                        |  8 ++++++-- 
>  meta/conf/bitbake.conf                              |  3 +++ 
>  meta/recipes-core/isar-bootstrap/isar-bootstrap.inc |  2 +- 
>  meta/recipes-kernel/linux/files/build-kernel.sh     | 10 +++++----- 
>  meta/recipes-kernel/linux/linux-custom.inc          |  1 + 
>  scripts/ci_build.sh                                 |  4 +++- 
>  9 files changed, 21 insertions(+), 14 deletions(-) 
>
> -- 
> 2.20.1 
>
>

[-- Attachment #1.2: Type: text/html, Size: 2241 bytes --]

Re: [PATCH v4 - misc fixes 0/6] miscellaneous (minor) fixes

[Baurzhan Ismagulov]

Hello Cedric,

On Fri, Nov 15, 2019 at 05:34:52AM -0800, chombourger@gmail.com wrote:
> any feedback against v4 of this patch series?

I replied yesterday, but the mail didn't hit the list indeed. Let me check.

With kind regards,
Baurzhan.

Re: [PATCH v4 - misc fixes 0/6] miscellaneous (minor) fixes

[Jan Kiszka]

On 15.11.19 14:34, chombourger@gmail.com wrote:
> 
> 
> On Friday, November 8, 2019 at 10:23:36 AM UTC+1, Cedric Hombourger wrote:
> 
>     The following minor fixes are being submitted for review and
>     inclusion to "next"
> 
> 
> any feedback against v4 of this patch series?

Just read patch 1 again and left a question. But I tested them all in a 
larger run and found no obvious issue.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux