[PATCH v3] wic-image: Fix up file ownership in isar repo after wic run

[PATCH v3] wic-image: Fix up file ownership in isar repo after wic run

[Jan Kiszka]

This prevents needlessly leaving the __pycache__ directories root-owned
behind after running wic as superuser - will only cause troubles to the
user when trying to clean up the isar repo.

We update the ownership only for selected subdirs of the repo to avoid
touching local files.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---

Changes in v3:
 - catch ALL relevant folders (first test, then post - hmpf)

 meta/classes/wic-img.bbclass | 1 +
 1 file changed, 1 insertion(+)

diff --git a/meta/classes/wic-img.bbclass b/meta/classes/wic-img.bbclass
index b56df7b..bc405c1 100644
--- a/meta/classes/wic-img.bbclass
+++ b/meta/classes/wic-img.bbclass
@@ -71,6 +71,7 @@ do_wic_image() {
     export MTOOLS_SKIP_CHECK=1
 
     sudo -E chroot ${BUILDCHROOT_DIR} ${ISARROOT}/scripts/wic create ${WKS_FILE} --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -o /tmp/ -e ${IMAGE_BASENAME} ${WIC_CREATE_EXTRA_ARGS}
+    sudo chown -R $(stat -c "%U" ${ISARROOT}) ${ISARROOT}/meta ${ISARROOT}/meta-isar ${ISARROOT}/scripts
     cp -f `ls -t -1 ${BUILDCHROOT_DIR}/tmp/${WKS_FILE}*.direct | head -1` ${WIC_IMAGE_FILE}
 }
 
-- 
2.16.4

Re: [PATCH v3] wic-image: Fix up file ownership in isar repo after wic run

[Christian Storm]

Hi Jan,

> This prevents needlessly leaving the __pycache__ directories root-owned
> behind after running wic as superuser - will only cause troubles to the
> user when trying to clean up the isar repo.

Hm, why not avoid creating those in the first place? I doubt that the
binary cache brings us that much performance benefit in our use cases.
Granted, you have to put/inject
  export PYTHONDONTWRITEBYTECODE=1
into *all* environments calling python so that not bothering with it at
all and removing/chown'ing them post-mortem might be easier or less
invasive.. Well....


Kind regards,
   Christian

-- 
Dr. Christian Storm
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Otto-Hahn-Ring 6, 81739 M�nchen, Germany

Re: [PATCH v3] wic-image: Fix up file ownership in isar repo after wic run

[Jan Kiszka]

On 2018-08-23 14:26, [ext] Christian Storm wrote:
> Hi Jan,
> 
>> This prevents needlessly leaving the __pycache__ directories root-owned
>> behind after running wic as superuser - will only cause troubles to the
>> user when trying to clean up the isar repo.
> 
> Hm, why not avoid creating those in the first place? I doubt that the
> binary cache brings us that much performance benefit in our use cases.
> Granted, you have to put/inject
>    export PYTHONDONTWRITEBYTECODE=1
> into *all* environments calling python so that not bothering with it at
> all and removing/chown'ing them post-mortem might be easier or less
> invasive.. Well....

Currently, that does not hurt (tried that first), but I wasn't sure 
about third-party plugins or future upstream stuff.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH v3] wic-image: Fix up file ownership in isar repo after wic run

[Maxim Yu. Osipov]

On 08/23/2018 03:12 PM, Jan Kiszka wrote:
> This prevents needlessly leaving the __pycache__ directories root-owned
> behind after running wic as superuser - will only cause troubles to the
> user when trying to clean up the isar repo.
> 
> We update the ownership only for selected subdirs of the repo to avoid
> touching local files.

Applied to the 'next',

Thanks,
Maxim.

> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
> 
> Changes in v3:
>   - catch ALL relevant folders (first test, then post - hmpf)
> 
>   meta/classes/wic-img.bbclass | 1 +
>   1 file changed, 1 insertion(+)
> 
> diff --git a/meta/classes/wic-img.bbclass b/meta/classes/wic-img.bbclass
> index b56df7b..bc405c1 100644
> --- a/meta/classes/wic-img.bbclass
> +++ b/meta/classes/wic-img.bbclass
> @@ -71,6 +71,7 @@ do_wic_image() {
>       export MTOOLS_SKIP_CHECK=1
>   
>       sudo -E chroot ${BUILDCHROOT_DIR} ${ISARROOT}/scripts/wic create ${WKS_FILE} --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -o /tmp/ -e ${IMAGE_BASENAME} ${WIC_CREATE_EXTRA_ARGS}
> +    sudo chown -R $(stat -c "%U" ${ISARROOT}) ${ISARROOT}/meta ${ISARROOT}/meta-isar ${ISARROOT}/scripts
>       cp -f `ls -t -1 ${BUILDCHROOT_DIR}/tmp/${WKS_FILE}*.direct | head -1` ${WIC_IMAGE_FILE}
>   }
>   
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH v3] wic-image: Fix up file ownership in isar repo after wic run

[Henning Schild]

Am Thu, 23 Aug 2018 14:12:19 +0200
schrieb Jan Kiszka <jan.kiszka@siemens.com>:

> This prevents needlessly leaving the __pycache__ directories
> root-owned behind after running wic as superuser - will only cause
> troubles to the user when trying to clean up the isar repo.
> 
> We update the ownership only for selected subdirs of the repo to avoid
> touching local files.
> 
> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
> 
> Changes in v3:
>  - catch ALL relevant folders (first test, then post - hmpf)
> 
>  meta/classes/wic-img.bbclass | 1 +
>  1 file changed, 1 insertion(+)
> 
> diff --git a/meta/classes/wic-img.bbclass
> b/meta/classes/wic-img.bbclass index b56df7b..bc405c1 100644
> --- a/meta/classes/wic-img.bbclass
> +++ b/meta/classes/wic-img.bbclass
> @@ -71,6 +71,7 @@ do_wic_image() {
>      export MTOOLS_SKIP_CHECK=1
>  
>      sudo -E chroot ${BUILDCHROOT_DIR} ${ISARROOT}/scripts/wic create
> ${WKS_FILE} --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -o /tmp/ -e
> ${IMAGE_BASENAME} ${WIC_CREATE_EXTRA_ARGS}
> +    sudo chown -R $(stat -c "%U" ${ISARROOT}) ${ISARROOT}/meta
> ${ISARROOT}/meta-isar ${ISARROOT}/scripts cp -f `ls -t -1
> ${BUILDCHROOT_DIR}/tmp/${WKS_FILE}*.direct | head -1`
> ${WIC_IMAGE_FILE} }

This breaks if ISARROOT is mounted ro, chown will fail with "read only
file system" and there will not even be any __pycache__ things to chown.

Steps to reproduce:
git clone isar
cd isar
mkdir out
docker run -v $(pwd):/this/:ro -v $(pwd)/out:/out:rw -e USER_ID=$(id
-u) --rm -t -i  --cap-add=SYS_ADMIN --cap-add=MKNOD -e
http_proxy=$http_proxy -e https_proxy=$https_proxy  -e
ftp_proxy=$ftp_proxy -e no_proxy=$no_proxy $CONTAINER sh -c bash
docker> cd /out
docker> source /this/isar-init-build-env
isar> bitbake <a wic image>

Henning

Re: [PATCH v3] wic-image: Fix up file ownership in isar repo after wic run

[Jan Kiszka]

On 12.09.18 14:49, Henning Schild wrote:
> Am Thu, 23 Aug 2018 14:12:19 +0200
> schrieb Jan Kiszka <jan.kiszka@siemens.com>:
> 
>> This prevents needlessly leaving the __pycache__ directories
>> root-owned behind after running wic as superuser - will only cause
>> troubles to the user when trying to clean up the isar repo.
>>
>> We update the ownership only for selected subdirs of the repo to avoid
>> touching local files.
>>
>> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
>> ---
>>
>> Changes in v3:
>>   - catch ALL relevant folders (first test, then post - hmpf)
>>
>>   meta/classes/wic-img.bbclass | 1 +
>>   1 file changed, 1 insertion(+)
>>
>> diff --git a/meta/classes/wic-img.bbclass
>> b/meta/classes/wic-img.bbclass index b56df7b..bc405c1 100644
>> --- a/meta/classes/wic-img.bbclass
>> +++ b/meta/classes/wic-img.bbclass
>> @@ -71,6 +71,7 @@ do_wic_image() {
>>       export MTOOLS_SKIP_CHECK=1
>>   
>>       sudo -E chroot ${BUILDCHROOT_DIR} ${ISARROOT}/scripts/wic create
>> ${WKS_FILE} --vars "${STAGING_DIR}/${MACHINE}/imgdata/" -o /tmp/ -e
>> ${IMAGE_BASENAME} ${WIC_CREATE_EXTRA_ARGS}
>> +    sudo chown -R $(stat -c "%U" ${ISARROOT}) ${ISARROOT}/meta
>> ${ISARROOT}/meta-isar ${ISARROOT}/scripts cp -f `ls -t -1
>> ${BUILDCHROOT_DIR}/tmp/${WKS_FILE}*.direct | head -1`
>> ${WIC_IMAGE_FILE} }
> 
> This breaks if ISARROOT is mounted ro, chown will fail with "read only
> file system" and there will not even be any __pycache__ things to chown.
> 
> Steps to reproduce:
> git clone isar
> cd isar
> mkdir out
> docker run -v $(pwd):/this/:ro -v $(pwd)/out:/out:rw -e USER_ID=$(id
> -u) --rm -t -i  --cap-add=SYS_ADMIN --cap-add=MKNOD -e
> http_proxy=$http_proxy -e https_proxy=$https_proxy  -e
> ftp_proxy=$ftp_proxy -e no_proxy=$no_proxy $CONTAINER sh -c bash
> docker> cd /out
> docker> source /this/isar-init-build-env
> isar> bitbake <a wic image>
> 

Valid remark, and I just realized that I recently hacked this effect away 
locally by adjusting some docker script.

Should be easy to fix by allowing that command to fail. Mind to write that 
one-liner and test it?

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux