[PATCH v2 0/2] test rebuilding hello series

[PATCH v2 0/2] test rebuilding hello series

[Maxim Yu. Osipov]

Hello,

This is a rebased v1 against current 'next'.

Fails on 'scripts/ci_build.sh -q'.

Regards,
Maxim.

Henning Schild (2):
  ci: test rebuilding "hello"
  ci: move IMAGE_INSTALL from config to ci script

 meta-isar/conf/multiconfig/qemuamd64-buster.conf | 2 --
 scripts/ci_build.sh                              | 8 ++++++++
 2 files changed, 8 insertions(+), 2 deletions(-)

-- 
2.11.0

[PATCH v2 1/2] ci: test rebuilding "hello"

[Maxim Yu. Osipov]

From: Henning Schild <henning.schild@siemens.com>

This patch adds the rebuilding of "hello" to one distro that does
not get rebuild from the cache. The recipe uses a feature which
currently does not work for the caching case.

Once we can cache "apt://" URIs the package should probably move to all
tests.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 scripts/ci_build.sh | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index bcc7545..b896f84 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -122,6 +122,11 @@ if [ -n "$REPRO_BUILD" ]; then
     sed -i -e 's/#ISAR_USE_CACHED_BASE_REPO ?= "1"/ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
 fi
 
+# "debian-buster" is currently not rebuild from the cache and the package
+# "hello" currently does not support being rebuild from the cache
+# so this distro is a good place to at least test the regular build
+echo "IMAGE_INSTALL_debian-buster += \"hello\"" >> conf/local.conf
+
 # Start build for the defined set of configurations
 bitbake $BB_ARGS $TARGETS_SET
 
-- 
2.11.0

[PATCH v2 2/2] ci: move IMAGE_INSTALL from config to ci script

[Maxim Yu. Osipov]

From: Henning Schild <henning.schild@siemens.com>

The intention was to create test coverage, not to include that package
into the default qemu target. So move the code around to keep coverage
but clean up the target.

Signed-off-by: Henning Schild <henning.schild@siemens.com>
---
 meta-isar/conf/multiconfig/qemuamd64-buster.conf | 2 --
 scripts/ci_build.sh                              | 3 +++
 2 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/meta-isar/conf/multiconfig/qemuamd64-buster.conf b/meta-isar/conf/multiconfig/qemuamd64-buster.conf
index 63df75c..394d76b 100644
--- a/meta-isar/conf/multiconfig/qemuamd64-buster.conf
+++ b/meta-isar/conf/multiconfig/qemuamd64-buster.conf
@@ -11,8 +11,6 @@ IMAGE_TYPE ?= "wic-img"
 WKS_FILE ?= "sdimage-efi"
 IMAGER_INSTALL += "${GRUB_BOOTLOADER_INSTALL}"
 
-IMAGE_INSTALL += "sshd-regen-keys"
-
 QEMU_ARCH ?= "x86_64"
 QEMU_MACHINE ?= "q35"
 QEMU_CPU ?= ""
diff --git a/scripts/ci_build.sh b/scripts/ci_build.sh
index b896f84..83d8fc9 100755
--- a/scripts/ci_build.sh
+++ b/scripts/ci_build.sh
@@ -122,6 +122,9 @@ if [ -n "$REPRO_BUILD" ]; then
     sed -i -e 's/#ISAR_USE_CACHED_BASE_REPO ?= "1"/ISAR_USE_CACHED_BASE_REPO ?= "1"/g' conf/local.conf
 fi
 
+# test some packages on just a few targets, for faster CI
+echo "IMAGE_INSTALL_debian-buster += \"sshd-regen-keys\"" >> conf/local.conf
+
 # "debian-buster" is currently not rebuild from the cache and the package
 # "hello" currently does not support being rebuild from the cache
 # so this distro is a good place to at least test the regular build
-- 
2.11.0

Re: [PATCH v2 0/2] test rebuilding hello series

[Henning Schild]

Hi Maxim,

why did you rebase and post that? Are you looking into the issue it
seems to reveal but not cause?

Henning

On Tue, 19 Feb 2019 07:37:34 +0100
"Maxim Yu. Osipov" <mosipov@ilbers.de> wrote:

> Hello,
> 
> This is a rebased v1 against current 'next'.
> 
> Fails on 'scripts/ci_build.sh -q'.
> 
> Regards,
> Maxim.
> 
> Henning Schild (2):
>   ci: test rebuilding "hello"
>   ci: move IMAGE_INSTALL from config to ci script
> 
>  meta-isar/conf/multiconfig/qemuamd64-buster.conf | 2 --
>  scripts/ci_build.sh                              | 8 ++++++++
>  2 files changed, 8 insertions(+), 2 deletions(-)
> 

Re: [PATCH v2 0/2] test rebuilding hello series

[Maxim Yu. Osipov]

Hi Henning,

I just rebased your series for convenience.

Your last email on this topic was on Wed, 13 Feb (see below)

Any progress on that?

As I said the problem is reproduced on my local PC - so this is
not jenkins environment problem.

Maxim.

-------- Forwarded Message --------
Subject: Re: weird CI errors
Date: Wed, 13 Feb 2019 11:07:17 +0100
From: Henning Schild <henning.schild@siemens.com>
To: Baurzhan Ismagulov <ibr@radix50.net>
CC: isar-users@googlegroups.com

On Wed, 13 Feb 2019 09:38:20 +0100
Baurzhan Ismagulov <ibr@radix50.net> wrote:

 > On Tue, Feb 12, 2019 at 07:12:16PM +0100, Henning Schild wrote:
 > > Turns out the CI runs into some weird issue that i can not
 > > reproduce on my machine. From a first couple of google hits it
 > > might have to do with network filesystem backing storage, might
 > > that be the case?
 >
 > The drive is local, no network fs.
 >
 >
 > > Error message:
 > > | shell-init: error retrieving current directory: getcwd: cannot
 > > access parent directories: Value too large for defined data type
 >
 > Which task? Google also shows that the directory might have been
 > deleted. Can we rule out races?

Might be a race, my local setup does not do parrallel multiconfig. I
think it was ${S} of the recipe hello. I do not see why any other task
would go ahead and remove/recreate that, but maybe some false sharing
between multiconfigs.

I will try if i can reproduce the issue by running the whole ci job. At
least on the jenkins it seem pretty reproducable.

Henning


On 2/19/19 4:20 PM, Henning Schild wrote:
> Hi Maxim,
> 
> why did you rebase and post that? Are you looking into the issue it
> seems to reveal but not cause?
> 
> Henning
> 
> On Tue, 19 Feb 2019 07:37:34 +0100
> "Maxim Yu. Osipov" <mosipov@ilbers.de> wrote:
> 
>> Hello,
>>
>> This is a rebased v1 against current 'next'.
>>
>> Fails on 'scripts/ci_build.sh -q'.
>>
>> Regards,
>> Maxim.
>>
>> Henning Schild (2):
>>    ci: test rebuilding "hello"
>>    ci: move IMAGE_INSTALL from config to ci script
>>
>>   meta-isar/conf/multiconfig/qemuamd64-buster.conf | 2 --
>>   scripts/ci_build.sh                              | 8 ++++++++
>>   2 files changed, 8 insertions(+), 2 deletions(-)
>>
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v2 0/2] test rebuilding hello series

[Henning Schild]

Am Wed, 20 Feb 2019 16:53:15 +0300
schrieb "Maxim Yu. Osipov" <mosipov@ilbers.de>:

> Hi Henning,
> 
> I just rebased your series for convenience.
> 
> Your last email on this topic was on Wed, 13 Feb (see below)
> 
> Any progress on that?
> 
> As I said the problem is reproduced on my local PC - so this is
> not jenkins environment problem.

I can reproduce it ... kind of. Because it is not always the same
error. I might just smuggle in the "hello" where the ssh key package is
at the moment, and forget about the fact that the example images should
maybe not contain these packages.

Henning

> Maxim.
> 
> -------- Forwarded Message --------
> Subject: Re: weird CI errors
> Date: Wed, 13 Feb 2019 11:07:17 +0100
> From: Henning Schild <henning.schild@siemens.com>
> To: Baurzhan Ismagulov <ibr@radix50.net>
> CC: isar-users@googlegroups.com
> 
> On Wed, 13 Feb 2019 09:38:20 +0100
> Baurzhan Ismagulov <ibr@radix50.net> wrote:
> 
>  > On Tue, Feb 12, 2019 at 07:12:16PM +0100, Henning Schild wrote:  
>  > > Turns out the CI runs into some weird issue that i can not
>  > > reproduce on my machine. From a first couple of google hits it
>  > > might have to do with network filesystem backing storage, might
>  > > that be the case?  
>  >
>  > The drive is local, no network fs.
>  >
>  >  
>  > > Error message:
>  > > | shell-init: error retrieving current directory: getcwd: cannot
>  > > access parent directories: Value too large for defined data
>  > > type  
>  >
>  > Which task? Google also shows that the directory might have been
>  > deleted. Can we rule out races?  
> 
> Might be a race, my local setup does not do parrallel multiconfig. I
> think it was ${S} of the recipe hello. I do not see why any other task
> would go ahead and remove/recreate that, but maybe some false sharing
> between multiconfigs.
> 
> I will try if i can reproduce the issue by running the whole ci job.
> At least on the jenkins it seem pretty reproducable.
> 
> Henning
> 
> 
> On 2/19/19 4:20 PM, Henning Schild wrote:
> > Hi Maxim,
> > 
> > why did you rebase and post that? Are you looking into the issue it
> > seems to reveal but not cause?
> > 
> > Henning
> > 
> > On Tue, 19 Feb 2019 07:37:34 +0100
> > "Maxim Yu. Osipov" <mosipov@ilbers.de> wrote:
> >   
> >> Hello,
> >>
> >> This is a rebased v1 against current 'next'.
> >>
> >> Fails on 'scripts/ci_build.sh -q'.
> >>
> >> Regards,
> >> Maxim.
> >>
> >> Henning Schild (2):
> >>    ci: test rebuilding "hello"
> >>    ci: move IMAGE_INSTALL from config to ci script
> >>
> >>   meta-isar/conf/multiconfig/qemuamd64-buster.conf | 2 --
> >>   scripts/ci_build.sh                              | 8 ++++++++
> >>   2 files changed, 8 insertions(+), 2 deletions(-)
> >>  
> >   
> 
>