[PATCH 0/1] Separate image rootfs for machines

[PATCH 0/1] Separate image rootfs for machines

[Maxim Yu. Osipov]

Hi everybody,

Image rootfs gets overwritten when running for two targets
with the same pair DISTRO and DISTRO_ARCH, this results start_vm script failure.
This patch fixes the problem.

Regards,
Maxim.

Maxim Yu. Osipov (1):
  classes/image: Separate image rootfs for machines

 meta/classes/image.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

-- 
2.11.0

[PATCH 1/1] classes/image: Separate image rootfs for machines

[Maxim Yu. Osipov]

Signed-off-by: Maxim Yu. Osipov <mosipov@ilbers.de>
---
 meta/classes/image.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index 2514c88..ed9d6eb 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -3,7 +3,7 @@
 
 IMAGE_INSTALL ?= ""
 IMAGE_TYPE    ?= "ext4-img"
-IMAGE_ROOTFS   = "${WORKDIR}/rootfs"
+IMAGE_ROOTFS   = "${WORKDIR}/${MACHINE}/rootfs/"
 
 IMAGE_INSTALL += "${@ ("linux-image-" + d.getVar("KERNEL_NAME", True)) if d.getVar("KERNEL_NAME", True) else ""}"
 
-- 
2.11.0

Re: [PATCH 1/1] classes/image: Separate image rootfs for machines

[Jan Kiszka]

On 11.03.19 18:03, Maxim Yu. Osipov wrote:
> Signed-off-by: Maxim Yu. Osipov <mosipov@ilbers.de>
> ---
>   meta/classes/image.bbclass | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index 2514c88..ed9d6eb 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -3,7 +3,7 @@
>   
>   IMAGE_INSTALL ?= ""
>   IMAGE_TYPE    ?= "ext4-img"
> -IMAGE_ROOTFS   = "${WORKDIR}/rootfs"
> +IMAGE_ROOTFS   = "${WORKDIR}/${MACHINE}/rootfs/"
>   
>   IMAGE_INSTALL += "${@ ("linux-image-" + d.getVar("KERNEL_NAME", True)) if d.getVar("KERNEL_NAME", True) else ""}"
>   
> 

Hmm, I wonder if that is actually a sufficient change - or a needed one. Just 
checked jailhouse-images, and there I do

PN .= "-${MACHINE}"

in the image recipe to ensure that distinct images (in different IMAGE_ROOTFS) 
are built. That change above will still run the recipe multiple times, 
overwriting at least the logs in confusing ways, if not more, no?

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH 1/1] classes/image: Separate image rootfs for machines

[cedric_hombourger]

[-- Attachment #1.1: Type: text/plain, Size: 349 bytes --]


>
> That change above will still run the recipe multiple times, 
> overwriting at least the logs in confusing ways, if not more, no? 
>

agreed. should we align with OE and have tmp/work/${DISTRO_ARCH}-${DISTRO}, 
tmp/work/all-${DISTRO} and tmp/work/${MACHINE}-${DISTRO} instead of having 
everything land under tmp/work/${DISTRO}-${DISTRO_ARCH}/?

[-- Attachment #1.2: Type: text/html, Size: 519 bytes --]

Re: [PATCH 1/1] classes/image: Separate image rootfs for machines

[Jan Kiszka]

On 11.03.19 20:43, cedric_hombourger@mentor.com wrote:
>     That change above will still run the recipe multiple times,
>     overwriting at least the logs in confusing ways, if not more, no?
> 
> 
> agreed. should we align with OE and have tmp/work/${DISTRO_ARCH}-${DISTRO}, 
> tmp/work/all-${DISTRO} and tmp/work/${MACHINE}-${DISTRO} instead of having 
> everything land under tmp/work/${DISTRO}-${DISTRO_ARCH}/?
> 

We just need to be careful not to destroy legitimate sharing of packages this 
way, e.g. if I build one kernel for multiple target boards. I think we are way 
ahead of OE in this regard with using multiconfig.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

[PATCH] isar-image: use a per-machine WORKDIR for image builds

[Cedric Hombourger]

Maxim Yu. Osipov reported that the "rootfs" directory created by
Isar to build images gets overwritten when running for two targets
with the same pair DISTRO and DISTRO_ARCH. It became obvious after
looking at start_vm script failures. Place all image build artifacts
in $DISTRO-$MACHINE instead of $DISTRO-$DISTRO_ARCH. This change
remains compatible with multiconfig builds (validated with a CI
build).

Reported-by: Maxim Yu. Osipov <mosipov@ilbers.de>
Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/classes/isar-image.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/isar-image.bbclass b/meta/classes/isar-image.bbclass
index cdd1651..a7f71a4 100644
--- a/meta/classes/isar-image.bbclass
+++ b/meta/classes/isar-image.bbclass
@@ -19,7 +19,7 @@ DEPENDS += "${IMAGE_INSTALL} ${IMAGE_TRANSIENT_PACKAGES}"
 
 IMAGE_TRANSIENT_PACKAGES += "isar-cfg-localepurge isar-cfg-rootpw"
 
-WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
+WORKDIR = "${TMPDIR}/work/${DISTRO}-${MACHINE}/${PN}"
 
 ISAR_RELEASE_CMD_DEFAULT = "git -C ${LAYERDIR_core} describe --tags --dirty --match 'v[0-9].[0-9]*'"
 ISAR_RELEASE_CMD ?= "${ISAR_RELEASE_CMD_DEFAULT}"
-- 
2.11.0

Re: [PATCH 1/1] classes/image: Separate image rootfs for machines

[Maxim Yu. Osipov]

On 3/11/19 7:00 PM, Jan Kiszka wrote:
> On 11.03.19 18:03, Maxim Yu. Osipov wrote:
>> Signed-off-by: Maxim Yu. Osipov <mosipov@ilbers.de>
>> ---
>>   meta/classes/image.bbclass | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
>> index 2514c88..ed9d6eb 100644
>> --- a/meta/classes/image.bbclass
>> +++ b/meta/classes/image.bbclass
>> @@ -3,7 +3,7 @@
>>   IMAGE_INSTALL ?= ""
>>   IMAGE_TYPE    ?= "ext4-img"
>> -IMAGE_ROOTFS   = "${WORKDIR}/rootfs"
>> +IMAGE_ROOTFS   = "${WORKDIR}/${MACHINE}/rootfs/"
>>   IMAGE_INSTALL += "${@ ("linux-image-" + d.getVar("KERNEL_NAME", 
>> True)) if d.getVar("KERNEL_NAME", True) else ""}"
>>
> 
> Hmm, I wonder if that is actually a sufficient change - or a needed one. 
> Just checked jailhouse-images, and there I do
> 
> PN .= "-${MACHINE}"
> 
> in the image recipe to ensure that distinct images (in different 
> IMAGE_ROOTFS) are built. That change above will still run the recipe 
> multiple times, overwriting at least the logs in confusing ways, if not 
> more, no?

Agree - this is the right approach.

Maxim.




-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH] isar-image: use a per-machine WORKDIR for image builds

[Maxim Yu. Osipov]

Hi Cedric,

1) I applied it and run 'ci_build -q -f' and build failed:

http://isar-build.org:8080/job/isar_mosipov_develop/52/

2) If we really want to move towards aligning with the OE as you suggest:

 >should we align with OE and have tmp/work/${DISTRO_ARCH}-${DISTRO}, 
 >tmp/work/all-${DISTRO} and tmp/work/${MACHINE}-${DISTRO} instead of
 > having everything land under tmp/work/${DISTRO}-${DISTRO_ARCH}/?

this is the only first and partial step.

This time I will follow the way suggested by Jan by appending ${MACHINE} 
to affected recipes.


Thanks,
Maxim.


On 3/12/19 6:57 AM, Cedric Hombourger wrote:
> Maxim Yu. Osipov reported that the "rootfs" directory created by
> Isar to build images gets overwritten when running for two targets
> with the same pair DISTRO and DISTRO_ARCH. It became obvious after
> looking at start_vm script failures. Place all image build artifacts
> in $DISTRO-$MACHINE instead of $DISTRO-$DISTRO_ARCH. This change
> remains compatible with multiconfig builds (validated with a CI
> build).
> 
> Reported-by: Maxim Yu. Osipov <mosipov@ilbers.de>
> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
> ---
>   meta/classes/isar-image.bbclass | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/classes/isar-image.bbclass b/meta/classes/isar-image.bbclass
> index cdd1651..a7f71a4 100644
> --- a/meta/classes/isar-image.bbclass
> +++ b/meta/classes/isar-image.bbclass
> @@ -19,7 +19,7 @@ DEPENDS += "${IMAGE_INSTALL} ${IMAGE_TRANSIENT_PACKAGES}"
>   
>   IMAGE_TRANSIENT_PACKAGES += "isar-cfg-localepurge isar-cfg-rootpw"
>   
> -WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
> +WORKDIR = "${TMPDIR}/work/${DISTRO}-${MACHINE}/${PN}"
>   
>   ISAR_RELEASE_CMD_DEFAULT = "git -C ${LAYERDIR_core} describe --tags --dirty --match 'v[0-9].[0-9]*'"
>   ISAR_RELEASE_CMD ?= "${ISAR_RELEASE_CMD_DEFAULT}"
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH 1/1] classes/image: Separate image rootfs for machines

[Baurzhan Ismagulov]

Hello Cedric,

On Mon, Mar 11, 2019 at 12:43:10PM -0700, cedric_hombourger@mentor.com wrote:
> agreed. should we align with OE and have tmp/work/${DISTRO_ARCH}-${DISTRO}, 
> tmp/work/all-${DISTRO} and tmp/work/${MACHINE}-${DISTRO} instead of having 
> everything land under tmp/work/${DISTRO}-${DISTRO_ARCH}/?

Thanks for a nice summary!

To make it clear: Maxim's current patch is a workaround fixing start_vm and CI
for the release. We do plan to fix the problem in a second step through
separating the work directories.

Regarding the structure, I find the separation
tmp/work/${DISTRO}-${DISTRO_ARCH}/${MACHINE} more intuitive, since one can
clearly see from the directory structure what belongs together:

* ${DISTRO}-${DISTRO_ARCH} + machine1
* ${DISTRO}-${DISTRO_ARCH} + machine2

rather than

* ${DISTRO_ARCH}-${DISTRO} + all-${DISTRO} + machine1-${DISTRO}
* ${DISTRO_ARCH}-${DISTRO} + all-${DISTRO} + machine2-${DISTRO}

Besides, all-${DISTRO} requires careful consideration, since OE really builds
them once by default, whereas we would build the _all.debs for every arch. That
is a good point, because we could optimize some bits here (dpkg-buildpackage
--build=all).

With kind regards,
Baurzhan.