From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6864480010557718528 Date: Sat, 5 Sep 2020 09:24:08 -0700 (PDT) From: "vijaikumar....@gmail.com" To: isar-users Message-Id: In-Reply-To: <20200905103057.2a95e915@md1za8fc.ad001.siemens.net> References: <20200902185624.15044-1-Vijaikumar_Kanagarajan@mentor.com> <20200902185624.15044-4-Vijaikumar_Kanagarajan@mentor.com> <20200905103057.2a95e915@md1za8fc.ad001.siemens.net> Subject: Re: [PATCH v2 03/10] wic-img: Satisfy the quirks of latest wic MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_628_1995415194.1599323048228" X-TUID: 8Y9jeM5dQSYv ------=_Part_628_1995415194.1599323048228 Content-Type: multipart/alternative; boundary="----=_Part_629_469055968.1599323048228" ------=_Part_629_469055968.1599323048228 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit On Saturday, September 5, 2020 at 2:00:59 PM UTC+5:30 Henning Schild wrote: > On Thu, 3 Sep 2020 00:26:17 +0530 > Vijai Kumar K wrote: > > > The latest WIC doesnot use fakeroot if > > pseudo_dir(IMAGE_ROOTFS/../pseudo) is non-existent. This results in > > wic_fakeroot not being used. Create a dummy pseudo_dir to satisfy wic. > > > > WIC hardcodes the python(/usr/bin/python3-native/python3) to be used > > while running bmaptool. The above path is OE specific and hence breaks > > build in ISAR. Create a symbolic link in the above location to satisfy > > wic. > > > > Signed-off-by: Vijai Kumar K > > --- > > meta/classes/wic-img.bbclass | 14 ++++++++++++-- > > 1 file changed, 12 insertions(+), 2 deletions(-) > > > > diff --git a/meta/classes/wic-img.bbclass > > b/meta/classes/wic-img.bbclass index afab394..a2c9627 100644 > > --- a/meta/classes/wic-img.bbclass > > +++ b/meta/classes/wic-img.bbclass > > @@ -143,6 +143,7 @@ EOSUDO > > export FAKEROOTCMD=${FAKEROOTCMD} > > export BUILDDIR=${BUILDDIR} > > export MTOOLS_SKIP_CHECK=1 > > + mkdir -p ${IMAGE_ROOTFS}/../pseudo > > > > # create the temp dir in the buildchroot to ensure uniqueness > > WICTMP=$(cd ${BUILDCHROOT_DIR}; mktemp -d -p tmp) > > @@ -157,22 +158,31 @@ EOSUDO > > IMAGE_FULLNAME="$6" > > IMAGE_BASENAME="$7" > > shift 7 > > - > > + # The python path is hard-coded as > > /usr/bin/python3-native/python3 in wic. Handle that. > > + mkdir -p /usr/bin/python3-native/ > > + if [ $(head -1 $(which bmaptool) | grep python3) ];then > > + ln -s /usr/bin/python3 /usr/bin/python3-native/python3 > > + else > > + ln -s /usr/bin/python2 /usr/bin/python3-native/python3 > > + fi > > Can this be moved to wic_fakeroot? That script was designed to hold all > the weird hacks ... > Unfortunately no. This invocation doesnot use fakeroot at all. Please see [1] & [2]. pseudo argument not used for exec_native_cmd which will in-turn not use fakeroot. [1] https://github.com/openembedded/openembedded-core/blob/404292b570a78895a1c7900eeb319e36e31dec20/scripts/lib/wic/plugins/imager/direct.py#L210 [2] https://github.com/openembedded/openembedded-core/blob/404292b570a78895a1c7900eeb319e36e31dec20/scripts/lib/wic/misc.py#L138 > I guess you need to "ln -sf" to make that whole thing executable > multiple times. > Sorry I did not get this completely. -f could be good to have to overwrite. We are remove this after wic runs so i didnot use -f here. > > Can you explain why we need the case where python2 is linked as python3? > I thought I captured it in commit message but looks like I have missed it. Because of stretch. bmaptools in stretch is python2 based and in buster and subsequent releases it is python3 based. OE-core in [1] has hardcoded to python3 since they have moved bmaptools to the latest one which uses python3. We would still need to workaround [1] for stretch too. Will capture this missed detail in commit message in V3. [1] https://github.com/openembedded/openembedded-core/blob/404292b570a78895a1c7900eeb319e36e31dec20/scripts/lib/wic/plugins/imager/direct.py#L210 > > export PATH="$ISARROOT/bitbake/bin:$PATH" > > "$ISARROOT"/scripts/wic create "$WKS_FULL_PATH" \ > > --vars "$STAGING_DIR/$MACHINE/imgdata/" \ > > -o "/$WICTMP/${IMAGE_FULLNAME}.wic/" \ > > --bmap \ > > - -e "$IMAGE_BASENAME" $@' \ > > + -e "$IMAGE_BASENAME" $@ > > + rm -rf /usr/bin/python3-native' \ > > This looks like the rm is in the middle of wic, should probably come > after. > It is at the end of my_script. Should be okay but bit difficult to read though. > > my_script "${ISARROOT}" "${WKS_FULL_PATH}" > > "${STAGING_DIR}" \ "${MACHINE}" "${WICTMP}" "${IMAGE_FULLNAME}" > > "${IMAGE_BASENAME}" \ ${WIC_CREATE_EXTRA_ARGS} > > + > > sudo chown -R $(stat -c "%U" ${ISARROOT}) ${ISARROOT}/meta > > ${ISARROOT}/meta-isar ${ISARROOT}/scripts || true WIC_DIRECT=$(ls -t > > -1 ${BUILDCHROOT_DIR}/$WICTMP/${IMAGE_FULLNAME}.wic/*.direct | head > > -1) sudo chown -R $(id -u):$(id -g) ${BUILDCHROOT_DIR}/${WICTMP} mv > > -f ${WIC_DIRECT} ${WIC_IMAGE_FILE} mv -f ${WIC_DIRECT}.bmap > > ${WIC_IMAGE_FILE}.bmap rm -rf ${BUILDCHROOT_DIR}/${WICTMP} > > + rm -rf ${IMAGE_ROOTFS}/../pseudo > > } > > > > do_wic_image[file-checksums] += "${WKS_FILE_CHECKSUM}" > > ------=_Part_629_469055968.1599323048228 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit

On Saturday, September 5, 2020 at 2:00:59 PM UTC+5:30 Henning Schild wrote:
On Thu, 3 Sep 2020 00:26:17 +0530
Vijai Kumar K <Vijaikumar_...@mentor.com> wrote:

> The latest WIC doesnot use fakeroot if
> pseudo_dir(IMAGE_ROOTFS/../pseudo) is non-existent. This results in
> wic_fakeroot not being used. Create a dummy pseudo_dir to satisfy wic.
>
> WIC hardcodes the python(/usr/bin/python3-native/python3) to be used
> while running bmaptool. The above path is OE specific and hence breaks
> build in ISAR. Create a symbolic link in the above location to satisfy
> wic.
>
> Signed-off-by: Vijai Kumar K <Vijaikumar_...@mentor.com>
> ---
> meta/classes/wic-img.bbclass | 14 ++++++++++++--
> 1 file changed, 12 insertions(+), 2 deletions(-)
>
> diff --git a/meta/classes/wic-img.bbclass
> b/meta/classes/wic-img.bbclass index afab394..a2c9627 100644
> --- a/meta/classes/wic-img.bbclass
> +++ b/meta/classes/wic-img.bbclass
> @@ -143,6 +143,7 @@ EOSUDO
> export FAKEROOTCMD=${FAKEROOTCMD}
> export BUILDDIR=${BUILDDIR}
> export MTOOLS_SKIP_CHECK=1
> + mkdir -p ${IMAGE_ROOTFS}/../pseudo
>
> # create the temp dir in the buildchroot to ensure uniqueness
> WICTMP=$(cd ${BUILDCHROOT_DIR}; mktemp -d -p tmp)
> @@ -157,22 +158,31 @@ EOSUDO
> IMAGE_FULLNAME="$6"
> IMAGE_BASENAME="$7"
> shift 7
> -
> + # The python path is hard-coded as
> /usr/bin/python3-native/python3 in wic. Handle that.
> + mkdir -p /usr/bin/python3-native/
> + if [ $(head -1 $(which bmaptool) | grep python3) ];then
> + ln -s /usr/bin/python3 /usr/bin/python3-native/python3
> + else
> + ln -s /usr/bin/python2 /usr/bin/python3-native/python3
> + fi

Can this be moved to wic_fakeroot? That script was designed to hold all
the weird hacks ...

Unfortunately no. This invocation doesnot use fakeroot at all. Please see [1] & [2]. pseudo argument not used for exec_native_cmd
which will in-turn not use fakeroot.

[1] https://github.com/openembedded/openembedded-core/blob/404292b570a78895a1c7900eeb319e36e31dec20/scripts/lib/wic/plugins/imager/direct.py#L210
[2] https://github.com/openembedded/openembedded-core/blob/404292b570a78895a1c7900eeb319e36e31dec20/scripts/lib/wic/misc.py#L138



I guess you need to "ln -sf" to make that whole thing executable
multiple times.

Sorry I did not get this completely. -f could be good to have to overwrite. We are remove this
after wic runs so i didnot use -f here.
 

Can you explain why we need the case where python2 is linked as python3?

I thought I captured it in commit message but looks like I have missed it.
Because of stretch. bmaptools in stretch is python2 based and in buster and
subsequent releases it is python3 based.
OE-core in [1] has hardcoded to python3 since they have moved bmaptools to the latest one
which uses python3. We would still need to workaround [1] for stretch too.

Will capture this missed detail in commit message in V3.

[1] https://github.com/openembedded/openembedded-core/blob/404292b570a78895a1c7900eeb319e36e31dec20/scripts/lib/wic/plugins/imager/direct.py#L210 


> export PATH="$ISARROOT/bitbake/bin:$PATH"
> "$ISARROOT"/scripts/wic create "$WKS_FULL_PATH" \
> --vars "$STAGING_DIR/$MACHINE/imgdata/" \
> -o "/$WICTMP/${IMAGE_FULLNAME}.wic/" \
> --bmap \
> - -e "$IMAGE_BASENAME" $@' \
> + -e "$IMAGE_BASENAME" $@
> + rm -rf /usr/bin/python3-native' \

This looks like the rm is in the middle of wic, should probably come
after.

It is at the end of my_script.  Should be okay but bit difficult to read though.


> my_script "${ISARROOT}" "${WKS_FULL_PATH}"
> "${STAGING_DIR}" \ "${MACHINE}" "${WICTMP}" "${IMAGE_FULLNAME}"
> "${IMAGE_BASENAME}" \ ${WIC_CREATE_EXTRA_ARGS}
> +
> sudo chown -R $(stat -c "%U" ${ISARROOT}) ${ISARROOT}/meta
> ${ISARROOT}/meta-isar ${ISARROOT}/scripts || true WIC_DIRECT=$(ls -t
> -1 ${BUILDCHROOT_DIR}/$WICTMP/${IMAGE_FULLNAME}.wic/*.direct | head
> -1) sudo chown -R $(id -u):$(id -g) ${BUILDCHROOT_DIR}/${WICTMP} mv
> -f ${WIC_DIRECT} ${WIC_IMAGE_FILE} mv -f ${WIC_DIRECT}.bmap
> ${WIC_IMAGE_FILE}.bmap rm -rf ${BUILDCHROOT_DIR}/${WICTMP}
> + rm -rf ${IMAGE_ROOTFS}/../pseudo
> }
>
> do_wic_image[file-checksums] += "${WKS_FILE_CHECKSUM}"

------=_Part_629_469055968.1599323048228-- ------=_Part_628_1995415194.1599323048228--