[PATCH v3 0/5] Unify work space for packages

public inbox for isar-users@googlegroups.com
 help / color / mirror / Atom feed

* [PATCH v3 0/5] Unify work space for packages
@ 2017-09-11 19:27 Alexander Smirnov
  2017-09-11 19:27 ` [PATCH v3 1/5] image: Unify path to image rootfs Alexander Smirnov
                   ` (5 more replies)
  0 siblings, 6 replies; 11+ messages in thread
From: Alexander Smirnov @ 2017-09-11 19:27 UTC (permalink / raw)
  To: isar-users; +Cc: Alexander Smirnov

Hi all,

this series does the following:
 - Simplifies folders structure for multiconfig.
 - Unifies work space for packages.
 - Use unified canonical names for  build artifacts.

The overal build tree with multiconfig looks like the following:
$ ls -1 tmp/work/
 debian-jessie-amd64
 debian-jessie-armhf
 debian-jessie-i386
 debian-stretch-amd64
 debian-stretch-armhf
 debian-stretch-i386
 debian-wheezy-armhf
 raspbian-jessie-armhf

Package workdir looks now like the following:
$ ls -1 tmp/work/debian-jessie-amd64/hello-0.1+g7f35942-1-r0/
git
hello_0.1+g7f35942-1_amd64.changes
hello_0.1+g7f35942-1_amd64.deb
hello_0.1+g7f35942-1.dsc
hello_0.1+g7f35942-1.tar.gz
temp

Changes since v2:
 - PATCH 1: Typo in commit message.
 - PATCH 2: Drop WORKDIR_RELATIVE and WORKDIR_NAME from bibtbake.conf,
            get relative path using inline python call,
            replace multiple sed call by single one.
 - PATCH 5: Update comment to be correct.

Changes since v1:
 - Split first patch into pieces.
 - Use variable in multistrap config.
 - Rename ext4 image to unified cacnonical name (patch 4).

With best regards,
Alex

Alexander Smirnov (5):
  image: Unify path to image rootfs
  isar: Change build folders tree
  stamps: Name stamps canonicaly and unified
  image: Name image canonicaly and unified
  class/dpkg: Unify workplace for packages

 meta-isar/classes/rpi-sdimg.bbclass              |  2 +-
 meta-isar/recipes-core/images/isar-image-base.bb | 18 ++++++++----------
 meta/classes/dpkg.bbclass                        | 15 +++++++++------
 meta/classes/ext4-img.bbclass                    | 12 ++++++------
 meta/classes/image.bbclass                       | 11 ++++++-----
 meta/conf/isar-bitbake.conf                      |  5 ++++-
 meta/recipes-devtools/buildchroot/buildchroot.bb |  8 ++++----
 scripts/start_vm                                 |  2 +-
 8 files changed, 39 insertions(+), 34 deletions(-)

-- 
2.1.4


^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH v3 1/5] image: Unify path to image rootfs
  2017-09-11 19:27 [PATCH v3 0/5] Unify work space for packages Alexander Smirnov
@ 2017-09-11 19:27 ` Alexander Smirnov
  2017-09-11 19:27 ` [PATCH v3 2/5] isar: Change build folders tree Alexander Smirnov
                   ` (4 subsequent siblings)
  5 siblings, 0 replies; 11+ messages in thread
From: Alexander Smirnov @ 2017-09-11 19:27 UTC (permalink / raw)
  To: isar-users; +Cc: Alexander Smirnov

Use dedicated variable to point to generated image root filesystem.

Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
---
 meta-isar/classes/rpi-sdimg.bbclass              | 2 +-
 meta-isar/recipes-core/images/isar-image-base.bb | 8 +++-----
 meta/classes/ext4-img.bbclass                    | 8 ++++----
 meta/classes/image.bbclass                       | 9 +++++----
 4 files changed, 13 insertions(+), 14 deletions(-)

diff --git a/meta-isar/classes/rpi-sdimg.bbclass b/meta-isar/classes/rpi-sdimg.bbclass
index 2614c41..91b09cd 100644
--- a/meta-isar/classes/rpi-sdimg.bbclass
+++ b/meta-isar/classes/rpi-sdimg.bbclass
@@ -47,7 +47,7 @@ do_rpi_sdimg () {
     BOOT_BLOCKS=$(LC_ALL=C parted -s ${SDIMG} unit b print | awk '/ 1 / { print substr($4, 1, length($4 -1)) / 512 /2 }')
     rm -f ${WORKDIR}/boot.img
     mkfs.vfat -n "${BOOTDD_VOLUME_ID}" -S 512 -C ${WORKDIR}/boot.img $BOOT_BLOCKS
-    mcopy -i ${WORKDIR}/boot.img -s ${S}/boot/* ::/
+    mcopy -i ${WORKDIR}/boot.img -s ${IMAGE_ROOTFS}/boot/* ::/
 
     # Burn Partitions
     dd if=${WORKDIR}/boot.img of=${SDIMG} conv=notrunc seek=1 bs=$(expr ${IMAGE_ROOTFS_ALIGNMENT} \* 1024) && sync && sync
diff --git a/meta-isar/recipes-core/images/isar-image-base.bb b/meta-isar/recipes-core/images/isar-image-base.bb
index b679d97..94cbc9e 100644
--- a/meta-isar/recipes-core/images/isar-image-base.bb
+++ b/meta-isar/recipes-core/images/isar-image-base.bb
@@ -18,8 +18,6 @@ IMAGE_PREINSTALL += "apt \
                      dbus"
 
 WORKDIR = "${TMPDIR}/work/${PN}/${MACHINE}/${DISTRO}"
-S = "${WORKDIR}/rootfs"
-IMAGE_ROOTFS = "${S}"
 
 do_rootfs[stamp-extra-info] = "${MACHINE}-${DISTRO}"
 
@@ -46,12 +44,12 @@ do_rootfs() {
     cd ${TOPDIR}
 
     # Create root filesystem
-    sudo multistrap -a ${DISTRO_ARCH} -d "${S}" -f "${WORKDIR}/multistrap.conf" || true
+    sudo multistrap -a ${DISTRO_ARCH} -d "${IMAGE_ROOTFS}" -f "${WORKDIR}/multistrap.conf" || true
 
     # Configure root filesystem
-    sudo chroot ${S} /configscript.sh ${MACHINE_SERIAL} ${BAUDRATE_TTY} \
+    sudo chroot ${IMAGE_ROOTFS} /configscript.sh ${MACHINE_SERIAL} ${BAUDRATE_TTY} \
         ${ROOTFS_DEV}
-    sudo rm ${S}/configscript.sh
+    sudo rm ${IMAGE_ROOTFS}/configscript.sh
 }
 
 addtask rootfs before do_populate
diff --git a/meta/classes/ext4-img.bbclass b/meta/classes/ext4-img.bbclass
index 8588626..f493037 100644
--- a/meta/classes/ext4-img.bbclass
+++ b/meta/classes/ext4-img.bbclass
@@ -14,23 +14,23 @@ do_ext4_image() {
 
     rm -f ${EXT4_IMAGE_FILE}
 
-    ROOTFS_SIZE=`sudo du -sm ${S} |  awk '{print $1 + ${ROOTFS_EXTRA};}'`
+    ROOTFS_SIZE=`sudo du -sm ${IMAGE_ROOTFS} |  awk '{print $1 + ${ROOTFS_EXTRA};}'`
     dd if=/dev/zero of=${EXT4_IMAGE_FILE} bs=1M count=${ROOTFS_SIZE}
 
     sudo mkfs.ext4 -F ${EXT4_IMAGE_FILE}
 
     mkdir -p ${WORKDIR}/mnt
     sudo mount -o loop ${EXT4_IMAGE_FILE} ${WORKDIR}/mnt
-    sudo cp -r ${S}/* ${WORKDIR}/mnt
+    sudo cp -r ${IMAGE_ROOTFS}/* ${WORKDIR}/mnt
     sudo umount ${WORKDIR}/mnt
     rm -r ${WORKDIR}/mnt
 
     if [ -n "${KERNEL_IMAGE}" ]; then
-        cp ${S}/boot/${KERNEL_IMAGE} ${DEPLOY_DIR_IMAGE}
+        cp ${IMAGE_ROOTFS}/boot/${KERNEL_IMAGE} ${DEPLOY_DIR_IMAGE}
     fi
 
     if [ -n "${INITRD_IMAGE}" ]; then
-        cp ${S}/boot/${INITRD_IMAGE} ${DEPLOY_DIR_IMAGE}
+        cp ${IMAGE_ROOTFS}/boot/${INITRD_IMAGE} ${DEPLOY_DIR_IMAGE}
     fi
 }
 
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index a7f0d74..dff2705 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -6,6 +6,7 @@ INITRD_IMAGE ?= ""
 
 IMAGE_INSTALL ?= ""
 IMAGE_TYPE    ?= "ext4-img"
+IMAGE_ROOTFS   = "${WORKDIR}/rootfs"
 
 inherit ${IMAGE_TYPE}
 
@@ -14,15 +15,15 @@ do_populate[stamp-extra-info] = "${MACHINE}-${DISTRO}"
 # Install Debian packages, that were built from sources
 do_populate() {
     if [ -n "${IMAGE_INSTALL}" ]; then
-        sudo mkdir -p ${S}/deb
+        sudo mkdir -p ${IMAGE_ROOTFS}/deb
 
         for p in ${IMAGE_INSTALL}; do
-            sudo cp ${DEPLOY_DIR_DEB}/${p}_*.deb ${S}/deb
+            sudo cp ${DEPLOY_DIR_DEB}/${p}_*.deb ${IMAGE_ROOTFS}/deb
         done
 
-        sudo chroot ${S} /usr/bin/dpkg -i -R /deb
+        sudo chroot ${IMAGE_ROOTFS} /usr/bin/dpkg -i -R /deb
 
-        sudo rm -rf ${S}/deb
+        sudo rm -rf ${IMAGE_ROOTFS}/deb
     fi
 }
 
-- 
2.1.4


^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH v3 2/5] isar: Change build folders tree
  2017-09-11 19:27 [PATCH v3 0/5] Unify work space for packages Alexander Smirnov
  2017-09-11 19:27 ` [PATCH v3 1/5] image: Unify path to image rootfs Alexander Smirnov
@ 2017-09-11 19:27 ` Alexander Smirnov
  2017-09-12  7:23   ` Claudius Heine
                     ` (2 more replies)
  2017-09-11 19:27 ` [PATCH v3 3/5] stamps: Name stamps canonicaly and unified Alexander Smirnov
                   ` (3 subsequent siblings)
  5 siblings, 3 replies; 11+ messages in thread
From: Alexander Smirnov @ 2017-09-11 19:27 UTC (permalink / raw)
  To: isar-users; +Cc: Alexander Smirnov

Differentiate folders tree, that are generated during build on
the highest possible level. So the overal tree now looks like the
following:

 tmp/work/${DISTRO}-${DISTRO_ARCH}/${PF}

This approach eliminates lots of subfolders to diffirentiate
buildchroot and image for multiconfig builds. Now each configuration
has private tree.

Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
---
 meta-isar/recipes-core/images/isar-image-base.bb | 26 +++++++++++++-----------
 meta/conf/isar-bitbake.conf                      |  3 ++-
 meta/recipes-devtools/buildchroot/buildchroot.bb | 26 +++++++++++++-----------
 3 files changed, 30 insertions(+), 25 deletions(-)

diff --git a/meta-isar/recipes-core/images/isar-image-base.bb b/meta-isar/recipes-core/images/isar-image-base.bb
index 94cbc9e..9e1b1da 100644
--- a/meta-isar/recipes-core/images/isar-image-base.bb
+++ b/meta-isar/recipes-core/images/isar-image-base.bb
@@ -17,7 +17,7 @@ DEPENDS += "${IMAGE_INSTALL}"
 IMAGE_PREINSTALL += "apt \
                      dbus"
 
-WORKDIR = "${TMPDIR}/work/${PN}/${MACHINE}/${DISTRO}"
+WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
 
 do_rootfs[stamp-extra-info] = "${MACHINE}-${DISTRO}"
 
@@ -30,18 +30,20 @@ do_rootfs() {
     install -m 755 ${THISDIR}/files/setup.sh ${WORKDIR}
     install -m 755 ${THISDIR}/files/download_dev-random ${WORKDIR}/hooks_multistrap/
 
-    # Adjust multistrap config
-    sed -i 's|##IMAGE_PREINSTALL##|${IMAGE_PREINSTALL}|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##DISTRO##|${DISTRO}|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##DISTRO_SUITE##|${DISTRO_SUITE}|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##CONFIG_SCRIPT##|./tmp/work/${PN}/${MACHINE}/${DISTRO}/configscript.sh|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##SETUP_SCRIPT##|./tmp/work/${PN}/${MACHINE}/${DISTRO}/setup.sh|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##DIR_HOOKS##|./tmp/work/${PN}/${MACHINE}/${DISTRO}/hooks_multistrap|' ${WORKDIR}/multistrap.conf
-
-    # Multistrap config use relative paths, so ensure that we are in the right folder
+    # Multistrap accepts only relative path in configuration files, so get it:
     cd ${TOPDIR}
+    WORKDIR_REL=$(python -c "import os.path; print os.path.relpath('${WORKDIR}')")
+
+    # Adjust multistrap config
+    sed -i -e 's|##IMAGE_PREINSTALL##|${IMAGE_PREINSTALL}|g' \
+           -e 's|##DISTRO##|${DISTRO}|g' \
+           -e 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|g' \
+           -e 's|##DISTRO_SUITE##|${DISTRO_SUITE}|g' \
+           -e 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|g' \
+           -e 's|##CONFIG_SCRIPT##|./'"$WORKDIR_REL"'/configscript.sh|g' \
+           -e 's|##SETUP_SCRIPT##|./'"$WORKDIR_REL"'/setup.sh|g' \
+           -e 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \
+              ${WORKDIR}/multistrap.conf
 
     # Create root filesystem
     sudo multistrap -a ${DISTRO_ARCH} -d "${IMAGE_ROOTFS}" -f "${WORKDIR}/multistrap.conf" || true
diff --git a/meta/conf/isar-bitbake.conf b/meta/conf/isar-bitbake.conf
index f85f5cc..5a26743 100644
--- a/meta/conf/isar-bitbake.conf
+++ b/meta/conf/isar-bitbake.conf
@@ -18,9 +18,10 @@
 # ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
 # OTHER DEALINGS IN THE SOFTWARE.
 
+WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PF}"
 DEPLOY_DIR_DEB = "${TMPDIR}/deploy/deb/${MACHINE}"
 SSTATE_DIR ?= "${TMPDIR}/sstate-cache"
-BUILDCHROOT_DIR = "${TOPDIR}/tmp/work/buildchroot/${DISTRO}-${DISTRO_ARCH}/rootfs"
+BUILDCHROOT_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot/rootfs"
 
 # Setup our default hash policy
 BB_SIGNATURE_HANDLER ?= "noop"
diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb
index ccba683..722007d 100644
--- a/meta/recipes-devtools/buildchroot/buildchroot.bb
+++ b/meta/recipes-devtools/buildchroot/buildchroot.bb
@@ -21,7 +21,7 @@ BUILDCHROOT_PREINSTALL ?= "gcc \
                            apt \
                            automake"
 
-WORKDIR = "${TMPDIR}/work/${PF}/${DISTRO}"
+WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
 
 do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
 
@@ -34,18 +34,20 @@ do_build() {
     install -m 755 ${THISDIR}/files/setup.sh ${WORKDIR}
     install -m 755 ${THISDIR}/files/download_dev-random ${WORKDIR}/hooks_multistrap/
 
-    # Adjust multistrap config
-    sed -i 's|##BUILDCHROOT_PREINSTALL##|${BUILDCHROOT_PREINSTALL}|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##DISTRO##|${DISTRO}|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##DISTRO_SUITE##|${DISTRO_SUITE}|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##CONFIG_SCRIPT##|./tmp/work/${PF}/${DISTRO}/configscript.sh|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##SETUP_SCRIPT##|./tmp/work/${PF}/${DISTRO}/setup.sh|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##DIR_HOOKS##|./tmp/work/${PF}/${DISTRO}/hooks_multistrap|' ${WORKDIR}/multistrap.conf
-
-    # Multistrap config use relative paths, so ensure that we are in the right folder
+    # Multistrap accepts only relative path in configuration files, so get it:
     cd ${TOPDIR}
+    WORKDIR_REL=$(python -c "import os.path; print os.path.relpath('${WORKDIR}')")
+
+    # Adjust multistrap config
+    sed -i -e 's|##IMAGE_PREINSTALL##|${IMAGE_PREINSTALL}|g' \
+           -e 's|##DISTRO##|${DISTRO}|g' \
+           -e 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|g' \
+           -e 's|##DISTRO_SUITE##|${DISTRO_SUITE}|g' \
+           -e 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|g' \
+           -e 's|##CONFIG_SCRIPT##|./'"$WORKDIR_REL"'/configscript.sh|g' \
+           -e 's|##SETUP_SCRIPT##|./'"$WORKDIR_REL"'/setup.sh|g' \
+           -e 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \
+              ${WORKDIR}/multistrap.conf
 
     # Create root filesystem
     sudo multistrap -a ${DISTRO_ARCH} -d "${BUILDCHROOT_DIR}" -f "${WORKDIR}/multistrap.conf" || true
-- 
2.1.4


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH v3 2/5] isar: Change build folders tree
  2017-09-11 19:27 ` [PATCH v3 2/5] isar: Change build folders tree Alexander Smirnov
@ 2017-09-12  7:23   ` Claudius Heine
  2017-09-12  7:57   ` Alexander Smirnov
  2017-09-12 12:52   ` Henning Schild
  2 siblings, 0 replies; 11+ messages in thread
From: Claudius Heine @ 2017-09-12  7:23 UTC (permalink / raw)
  To: Alexander Smirnov, isar-users

Hi Alex,

On 09/11/2017 09:27 PM, Alexander Smirnov wrote:
> diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb
> index ccba683..722007d 100644
> --- a/meta/recipes-devtools/buildchroot/buildchroot.bb
> +++ b/meta/recipes-devtools/buildchroot/buildchroot.bb
> @@ -21,7 +21,7 @@ BUILDCHROOT_PREINSTALL ?= "gcc \
>                              apt \
>                              automake"
>   
> -WORKDIR = "${TMPDIR}/work/${PF}/${DISTRO}"
> +WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
>   
>   do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>   
> @@ -34,18 +34,20 @@ do_build() {
>       install -m 755 ${THISDIR}/files/setup.sh ${WORKDIR}
>       install -m 755 ${THISDIR}/files/download_dev-random ${WORKDIR}/hooks_multistrap/
>   
> -    # Adjust multistrap config
> -    sed -i 's|##BUILDCHROOT_PREINSTALL##|${BUILDCHROOT_PREINSTALL}|' ${WORKDIR}/multistrap.conf
> -    sed -i 's|##DISTRO##|${DISTRO}|' ${WORKDIR}/multistrap.conf
> -    sed -i 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|' ${WORKDIR}/multistrap.conf
> -    sed -i 's|##DISTRO_SUITE##|${DISTRO_SUITE}|' ${WORKDIR}/multistrap.conf
> -    sed -i 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|' ${WORKDIR}/multistrap.conf
> -    sed -i 's|##CONFIG_SCRIPT##|./tmp/work/${PF}/${DISTRO}/configscript.sh|' ${WORKDIR}/multistrap.conf
> -    sed -i 's|##SETUP_SCRIPT##|./tmp/work/${PF}/${DISTRO}/setup.sh|' ${WORKDIR}/multistrap.conf
> -    sed -i 's|##DIR_HOOKS##|./tmp/work/${PF}/${DISTRO}/hooks_multistrap|' ${WORKDIR}/multistrap.conf
> -
> -    # Multistrap config use relative paths, so ensure that we are in the right folder
> +    # Multistrap accepts only relative path in configuration files, so get it:
>       cd ${TOPDIR}
> +    WORKDIR_REL=$(python -c "import os.path; print os.path.relpath('${WORKDIR}')")
Better. But have you tried something like this:

     WORKDIR_REL="${@ os.path.relpath(d.getVar(WORKDIR, True))}"

Since bitbake allows python inline code and the 'os' module should be 
imported per default it should work.

> +
> +    # Adjust multistrap config
> +    sed -i -e 's|##IMAGE_PREINSTALL##|${IMAGE_PREINSTALL}|g' \
> +           -e 's|##DISTRO##|${DISTRO}|g' \
> +           -e 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|g' \
> +           -e 's|##DISTRO_SUITE##|${DISTRO_SUITE}|g' \
> +           -e 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|g' \
> +           -e 's|##CONFIG_SCRIPT##|./'"$WORKDIR_REL"'/configscript.sh|g' \
> +           -e 's|##SETUP_SCRIPT##|./'"$WORKDIR_REL"'/setup.sh|g' \
> +           -e 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \
> +              ${WORKDIR}/multistrap.conf

Thanks for making it so easier for my eyes :)

>   
>       # Create root filesystem
>       sudo multistrap -a ${DISTRO_ARCH} -d "${BUILDCHROOT_DIR}" -f "${WORKDIR}/multistrap.conf" || true
> 

Cheers,
Claudius

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de

^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH v3 2/5] isar: Change build folders tree
  2017-09-11 19:27 ` [PATCH v3 2/5] isar: Change build folders tree Alexander Smirnov
  2017-09-12  7:23   ` Claudius Heine
@ 2017-09-12  7:57   ` Alexander Smirnov
  2017-09-12 12:52   ` Henning Schild
  2 siblings, 0 replies; 11+ messages in thread
From: Alexander Smirnov @ 2017-09-12  7:57 UTC (permalink / raw)
  To: isar-users; +Cc: Alexander Smirnov

Differentiate folders tree, that are generated during build on
the highest possible level. So the overal tree now looks like the
following:

 tmp/work/${DISTRO}-${DISTRO_ARCH}/${PF}

This approach eliminates lots of subfolders to diffirentiate
buildchroot and image for multiconfig builds. Now each configuration
has private tree.

Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
---
 meta-isar/recipes-core/images/isar-image-base.bb | 26 +++++++++++++-----------
 meta/conf/isar-bitbake.conf                      |  3 ++-
 meta/recipes-devtools/buildchroot/buildchroot.bb | 26 +++++++++++++-----------
 3 files changed, 30 insertions(+), 25 deletions(-)

diff --git a/meta-isar/recipes-core/images/isar-image-base.bb b/meta-isar/recipes-core/images/isar-image-base.bb
index 94cbc9e..5cb8b1c 100644
--- a/meta-isar/recipes-core/images/isar-image-base.bb
+++ b/meta-isar/recipes-core/images/isar-image-base.bb
@@ -17,7 +17,7 @@ DEPENDS += "${IMAGE_INSTALL}"
 IMAGE_PREINSTALL += "apt \
                      dbus"
 
-WORKDIR = "${TMPDIR}/work/${PN}/${MACHINE}/${DISTRO}"
+WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
 
 do_rootfs[stamp-extra-info] = "${MACHINE}-${DISTRO}"
 
@@ -30,18 +30,20 @@ do_rootfs() {
     install -m 755 ${THISDIR}/files/setup.sh ${WORKDIR}
     install -m 755 ${THISDIR}/files/download_dev-random ${WORKDIR}/hooks_multistrap/
 
-    # Adjust multistrap config
-    sed -i 's|##IMAGE_PREINSTALL##|${IMAGE_PREINSTALL}|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##DISTRO##|${DISTRO}|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##DISTRO_SUITE##|${DISTRO_SUITE}|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##CONFIG_SCRIPT##|./tmp/work/${PN}/${MACHINE}/${DISTRO}/configscript.sh|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##SETUP_SCRIPT##|./tmp/work/${PN}/${MACHINE}/${DISTRO}/setup.sh|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##DIR_HOOKS##|./tmp/work/${PN}/${MACHINE}/${DISTRO}/hooks_multistrap|' ${WORKDIR}/multistrap.conf
-
-    # Multistrap config use relative paths, so ensure that we are in the right folder
+    # Multistrap accepts only relative path in configuration files, so get it:
     cd ${TOPDIR}
+    WORKDIR_REL=${@ os.path.relpath(d.getVar("WORKDIR", True))}
+
+    # Adjust multistrap config
+    sed -i -e 's|##IMAGE_PREINSTALL##|${IMAGE_PREINSTALL}|g' \
+           -e 's|##DISTRO##|${DISTRO}|g' \
+           -e 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|g' \
+           -e 's|##DISTRO_SUITE##|${DISTRO_SUITE}|g' \
+           -e 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|g' \
+           -e 's|##CONFIG_SCRIPT##|./'"$WORKDIR_REL"'/configscript.sh|g' \
+           -e 's|##SETUP_SCRIPT##|./'"$WORKDIR_REL"'/setup.sh|g' \
+           -e 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \
+              ${WORKDIR}/multistrap.conf
 
     # Create root filesystem
     sudo multistrap -a ${DISTRO_ARCH} -d "${IMAGE_ROOTFS}" -f "${WORKDIR}/multistrap.conf" || true
diff --git a/meta/conf/isar-bitbake.conf b/meta/conf/isar-bitbake.conf
index f85f5cc..5a26743 100644
--- a/meta/conf/isar-bitbake.conf
+++ b/meta/conf/isar-bitbake.conf
@@ -18,9 +18,10 @@
 # ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
 # OTHER DEALINGS IN THE SOFTWARE.
 
+WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PF}"
 DEPLOY_DIR_DEB = "${TMPDIR}/deploy/deb/${MACHINE}"
 SSTATE_DIR ?= "${TMPDIR}/sstate-cache"
-BUILDCHROOT_DIR = "${TOPDIR}/tmp/work/buildchroot/${DISTRO}-${DISTRO_ARCH}/rootfs"
+BUILDCHROOT_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot/rootfs"
 
 # Setup our default hash policy
 BB_SIGNATURE_HANDLER ?= "noop"
diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb
index ccba683..30039f2 100644
--- a/meta/recipes-devtools/buildchroot/buildchroot.bb
+++ b/meta/recipes-devtools/buildchroot/buildchroot.bb
@@ -21,7 +21,7 @@ BUILDCHROOT_PREINSTALL ?= "gcc \
                            apt \
                            automake"
 
-WORKDIR = "${TMPDIR}/work/${PF}/${DISTRO}"
+WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
 
 do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
 
@@ -34,18 +34,20 @@ do_build() {
     install -m 755 ${THISDIR}/files/setup.sh ${WORKDIR}
     install -m 755 ${THISDIR}/files/download_dev-random ${WORKDIR}/hooks_multistrap/
 
-    # Adjust multistrap config
-    sed -i 's|##BUILDCHROOT_PREINSTALL##|${BUILDCHROOT_PREINSTALL}|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##DISTRO##|${DISTRO}|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##DISTRO_SUITE##|${DISTRO_SUITE}|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##CONFIG_SCRIPT##|./tmp/work/${PF}/${DISTRO}/configscript.sh|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##SETUP_SCRIPT##|./tmp/work/${PF}/${DISTRO}/setup.sh|' ${WORKDIR}/multistrap.conf
-    sed -i 's|##DIR_HOOKS##|./tmp/work/${PF}/${DISTRO}/hooks_multistrap|' ${WORKDIR}/multistrap.conf
-
-    # Multistrap config use relative paths, so ensure that we are in the right folder
+    # Multistrap accepts only relative path in configuration files, so get it:
     cd ${TOPDIR}
+    WORKDIR_REL=${@ os.path.relpath(d.getVar("WORKDIR", True))}
+
+    # Adjust multistrap config
+    sed -i -e 's|##BUILDCHROOT_PREINSTALL##|${BUILDCHROOT_PREINSTALL}|g' \
+           -e 's|##DISTRO##|${DISTRO}|g' \
+           -e 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|g' \
+           -e 's|##DISTRO_SUITE##|${DISTRO_SUITE}|g' \
+           -e 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|g' \
+           -e 's|##CONFIG_SCRIPT##|./'"$WORKDIR_REL"'/configscript.sh|g' \
+           -e 's|##SETUP_SCRIPT##|./'"$WORKDIR_REL"'/setup.sh|g' \
+           -e 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \
+              ${WORKDIR}/multistrap.conf
 
     # Create root filesystem
     sudo multistrap -a ${DISTRO_ARCH} -d "${BUILDCHROOT_DIR}" -f "${WORKDIR}/multistrap.conf" || true
-- 
2.1.4


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH v3 2/5] isar: Change build folders tree
  2017-09-11 19:27 ` [PATCH v3 2/5] isar: Change build folders tree Alexander Smirnov
  2017-09-12  7:23   ` Claudius Heine
  2017-09-12  7:57   ` Alexander Smirnov
@ 2017-09-12 12:52   ` Henning Schild
  2017-09-12 13:02     ` Alexander Smirnov
  2 siblings, 1 reply; 11+ messages in thread
From: Henning Schild @ 2017-09-12 12:52 UTC (permalink / raw)
  To: Alexander Smirnov; +Cc: isar-users

Am Mon, 11 Sep 2017 22:27:35 +0300
schrieb Alexander Smirnov <asmirnov@ilbers.de>:

> Differentiate folders tree, that are generated during build on
> the highest possible level. So the overal tree now looks like the
> following:
> 
>  tmp/work/${DISTRO}-${DISTRO_ARCH}/${PF}
> 
> This approach eliminates lots of subfolders to diffirentiate
> buildchroot and image for multiconfig builds. Now each configuration
> has private tree.
> 
> Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
> ---
>  meta-isar/recipes-core/images/isar-image-base.bb | 26
> +++++++++++++-----------
> meta/conf/isar-bitbake.conf                      |  3 ++-
> meta/recipes-devtools/buildchroot/buildchroot.bb | 26
> +++++++++++++----------- 3 files changed, 30 insertions(+), 25
> deletions(-)
> 
> diff --git a/meta-isar/recipes-core/images/isar-image-base.bb
> b/meta-isar/recipes-core/images/isar-image-base.bb index
> 94cbc9e..9e1b1da 100644 ---
> a/meta-isar/recipes-core/images/isar-image-base.bb +++
> b/meta-isar/recipes-core/images/isar-image-base.bb @@ -17,7 +17,7 @@
> DEPENDS += "${IMAGE_INSTALL}" IMAGE_PREINSTALL += "apt \
>                       dbus"
>  
> -WORKDIR = "${TMPDIR}/work/${PN}/${MACHINE}/${DISTRO}"
> +WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
>  
>  do_rootfs[stamp-extra-info] = "${MACHINE}-${DISTRO}"
>  
> @@ -30,18 +30,20 @@ do_rootfs() {
>      install -m 755 ${THISDIR}/files/setup.sh ${WORKDIR}
>      install -m 755 ${THISDIR}/files/download_dev-random
> ${WORKDIR}/hooks_multistrap/ 
> -    # Adjust multistrap config
> -    sed -i 's|##IMAGE_PREINSTALL##|${IMAGE_PREINSTALL}|'
> ${WORKDIR}/multistrap.conf
> -    sed -i 's|##DISTRO##|${DISTRO}|' ${WORKDIR}/multistrap.conf
> -    sed -i 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|'
> ${WORKDIR}/multistrap.conf
> -    sed -i 's|##DISTRO_SUITE##|${DISTRO_SUITE}|'
> ${WORKDIR}/multistrap.conf
> -    sed -i 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|'
> ${WORKDIR}/multistrap.conf
> -    sed -i
> 's|##CONFIG_SCRIPT##|./tmp/work/${PN}/${MACHINE}/${DISTRO}/configscript.sh|'
> ${WORKDIR}/multistrap.conf
> -    sed -i
> 's|##SETUP_SCRIPT##|./tmp/work/${PN}/${MACHINE}/${DISTRO}/setup.sh|'
> ${WORKDIR}/multistrap.conf
> -    sed -i
> 's|##DIR_HOOKS##|./tmp/work/${PN}/${MACHINE}/${DISTRO}/hooks_multistrap|'
> ${WORKDIR}/multistrap.conf -
> -    # Multistrap config use relative paths, so ensure that we are in
> the right folder
> +    # Multistrap accepts only relative path in configuration files,
> so get it: cd ${TOPDIR}
> +    WORKDIR_REL=$(python -c "import os.path; print
> os.path.relpath('${WORKDIR}')") +
> +    # Adjust multistrap config
> +    sed -i -e 's|##IMAGE_PREINSTALL##|${IMAGE_PREINSTALL}|g' \
> +           -e 's|##DISTRO##|${DISTRO}|g' \
> +           -e 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|g' \
> +           -e 's|##DISTRO_SUITE##|${DISTRO_SUITE}|g' \
> +           -e 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|g' \
> +           -e
> 's|##CONFIG_SCRIPT##|./'"$WORKDIR_REL"'/configscript.sh|g' \
> +           -e 's|##SETUP_SCRIPT##|./'"$WORKDIR_REL"'/setup.sh|g' \
> +           -e
> 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \
> +              ${WORKDIR}/multistrap.conf
>  
>      # Create root filesystem
>      sudo multistrap -a ${DISTRO_ARCH} -d "${IMAGE_ROOTFS}" -f
> "${WORKDIR}/multistrap.conf" || true diff --git
> a/meta/conf/isar-bitbake.conf b/meta/conf/isar-bitbake.conf index
> f85f5cc..5a26743 100644 --- a/meta/conf/isar-bitbake.conf
> +++ b/meta/conf/isar-bitbake.conf
> @@ -18,9 +18,10 @@
>  # ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE
> OR # OTHER DEALINGS IN THE SOFTWARE.
>  
> +WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PF}"
>  DEPLOY_DIR_DEB = "${TMPDIR}/deploy/deb/${MACHINE}"
>  SSTATE_DIR ?= "${TMPDIR}/sstate-cache"
> -BUILDCHROOT_DIR =
> "${TOPDIR}/tmp/work/buildchroot/${DISTRO}-${DISTRO_ARCH}/rootfs"
> +BUILDCHROOT_DIR =
> "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot/rootfs" # Setup
> our default hash policy BB_SIGNATURE_HANDLER ?= "noop"
> diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb
> b/meta/recipes-devtools/buildchroot/buildchroot.bb index
> ccba683..722007d 100644 ---
> a/meta/recipes-devtools/buildchroot/buildchroot.bb +++
> b/meta/recipes-devtools/buildchroot/buildchroot.bb @@ -21,7 +21,7 @@
> BUILDCHROOT_PREINSTALL ?= "gcc \ apt \
>                             automake"
>  
> -WORKDIR = "${TMPDIR}/work/${PF}/${DISTRO}"
> +WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
>  
>  do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>  
> @@ -34,18 +34,20 @@ do_build() {
>      install -m 755 ${THISDIR}/files/setup.sh ${WORKDIR}
>      install -m 755 ${THISDIR}/files/download_dev-random
> ${WORKDIR}/hooks_multistrap/ 
> -    # Adjust multistrap config
> -    sed -i 's|##BUILDCHROOT_PREINSTALL##|${BUILDCHROOT_PREINSTALL}|'
> ${WORKDIR}/multistrap.conf
> -    sed -i 's|##DISTRO##|${DISTRO}|' ${WORKDIR}/multistrap.conf
> -    sed -i 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|'
> ${WORKDIR}/multistrap.conf
> -    sed -i 's|##DISTRO_SUITE##|${DISTRO_SUITE}|'
> ${WORKDIR}/multistrap.conf
> -    sed -i 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|'
> ${WORKDIR}/multistrap.conf
> -    sed -i
> 's|##CONFIG_SCRIPT##|./tmp/work/${PF}/${DISTRO}/configscript.sh|'
> ${WORKDIR}/multistrap.conf
> -    sed -i 's|##SETUP_SCRIPT##|./tmp/work/${PF}/${DISTRO}/setup.sh|'
> ${WORKDIR}/multistrap.conf
> -    sed -i
> 's|##DIR_HOOKS##|./tmp/work/${PF}/${DISTRO}/hooks_multistrap|'
> ${WORKDIR}/multistrap.conf -
> -    # Multistrap config use relative paths, so ensure that we are in
> the right folder
> +    # Multistrap accepts only relative path in configuration files,
> so get it: cd ${TOPDIR}
> +    WORKDIR_REL=$(python -c "import os.path; print
> os.path.relpath('${WORKDIR}')") +
> +    # Adjust multistrap config
> +    sed -i -e 's|##IMAGE_PREINSTALL##|${IMAGE_PREINSTALL}|g' \
> +           -e 's|##DISTRO##|${DISTRO}|g' \
> +           -e 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|g' \
> +           -e 's|##DISTRO_SUITE##|${DISTRO_SUITE}|g' \
> +           -e 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|g' \
> +           -e
> 's|##CONFIG_SCRIPT##|./'"$WORKDIR_REL"'/configscript.sh|g' \
> +           -e 's|##SETUP_SCRIPT##|./'"$WORKDIR_REL"'/setup.sh|g' \
> +           -e
> 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \
> +              ${WORKDIR}/multistrap.conf

Arghh ... So easy on the eye and still wrong. Such style changes should
be in another patch, but i guess i started it.

You remove ##BUILDCHROOT_PREINSTALL## and add ##IMAGE_PREINSTALL##.

Henning

>      # Create root filesystem
>      sudo multistrap -a ${DISTRO_ARCH} -d "${BUILDCHROOT_DIR}" -f
> "${WORKDIR}/multistrap.conf" || true


^ permalink raw reply	[flat|nested] 11+ messages in thread

* Re: [PATCH v3 2/5] isar: Change build folders tree
  2017-09-12 12:52   ` Henning Schild
@ 2017-09-12 13:02     ` Alexander Smirnov
  0 siblings, 0 replies; 11+ messages in thread
From: Alexander Smirnov @ 2017-09-12 13:02 UTC (permalink / raw)
  To: Henning Schild; +Cc: isar-users



On 09/12/2017 03:52 PM, Henning Schild wrote:
> Am Mon, 11 Sep 2017 22:27:35 +0300
> schrieb Alexander Smirnov <asmirnov@ilbers.de>:
> 
>> Differentiate folders tree, that are generated during build on
>> the highest possible level. So the overal tree now looks like the
>> following:
>>
>>   tmp/work/${DISTRO}-${DISTRO_ARCH}/${PF}
>>
>> This approach eliminates lots of subfolders to diffirentiate
>> buildchroot and image for multiconfig builds. Now each configuration
>> has private tree.
>>
>> Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
>> ---
>>   meta-isar/recipes-core/images/isar-image-base.bb | 26
>> +++++++++++++-----------
>> meta/conf/isar-bitbake.conf                      |  3 ++-
>> meta/recipes-devtools/buildchroot/buildchroot.bb | 26
>> +++++++++++++----------- 3 files changed, 30 insertions(+), 25
>> deletions(-)
>>
>> diff --git a/meta-isar/recipes-core/images/isar-image-base.bb
>> b/meta-isar/recipes-core/images/isar-image-base.bb index
>> 94cbc9e..9e1b1da 100644 ---
>> a/meta-isar/recipes-core/images/isar-image-base.bb +++
>> b/meta-isar/recipes-core/images/isar-image-base.bb @@ -17,7 +17,7 @@
>> DEPENDS += "${IMAGE_INSTALL}" IMAGE_PREINSTALL += "apt \
>>                        dbus"
>>   
>> -WORKDIR = "${TMPDIR}/work/${PN}/${MACHINE}/${DISTRO}"
>> +WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
>>   
>>   do_rootfs[stamp-extra-info] = "${MACHINE}-${DISTRO}"
>>   
>> @@ -30,18 +30,20 @@ do_rootfs() {
>>       install -m 755 ${THISDIR}/files/setup.sh ${WORKDIR}
>>       install -m 755 ${THISDIR}/files/download_dev-random
>> ${WORKDIR}/hooks_multistrap/
>> -    # Adjust multistrap config
>> -    sed -i 's|##IMAGE_PREINSTALL##|${IMAGE_PREINSTALL}|'
>> ${WORKDIR}/multistrap.conf
>> -    sed -i 's|##DISTRO##|${DISTRO}|' ${WORKDIR}/multistrap.conf
>> -    sed -i 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|'
>> ${WORKDIR}/multistrap.conf
>> -    sed -i 's|##DISTRO_SUITE##|${DISTRO_SUITE}|'
>> ${WORKDIR}/multistrap.conf
>> -    sed -i 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|'
>> ${WORKDIR}/multistrap.conf
>> -    sed -i
>> 's|##CONFIG_SCRIPT##|./tmp/work/${PN}/${MACHINE}/${DISTRO}/configscript.sh|'
>> ${WORKDIR}/multistrap.conf
>> -    sed -i
>> 's|##SETUP_SCRIPT##|./tmp/work/${PN}/${MACHINE}/${DISTRO}/setup.sh|'
>> ${WORKDIR}/multistrap.conf
>> -    sed -i
>> 's|##DIR_HOOKS##|./tmp/work/${PN}/${MACHINE}/${DISTRO}/hooks_multistrap|'
>> ${WORKDIR}/multistrap.conf -
>> -    # Multistrap config use relative paths, so ensure that we are in
>> the right folder
>> +    # Multistrap accepts only relative path in configuration files,
>> so get it: cd ${TOPDIR}
>> +    WORKDIR_REL=$(python -c "import os.path; print
>> os.path.relpath('${WORKDIR}')") +
>> +    # Adjust multistrap config
>> +    sed -i -e 's|##IMAGE_PREINSTALL##|${IMAGE_PREINSTALL}|g' \
>> +           -e 's|##DISTRO##|${DISTRO}|g' \
>> +           -e 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|g' \
>> +           -e 's|##DISTRO_SUITE##|${DISTRO_SUITE}|g' \
>> +           -e 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|g' \
>> +           -e
>> 's|##CONFIG_SCRIPT##|./'"$WORKDIR_REL"'/configscript.sh|g' \
>> +           -e 's|##SETUP_SCRIPT##|./'"$WORKDIR_REL"'/setup.sh|g' \
>> +           -e
>> 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \
>> +              ${WORKDIR}/multistrap.conf
>>   
>>       # Create root filesystem
>>       sudo multistrap -a ${DISTRO_ARCH} -d "${IMAGE_ROOTFS}" -f
>> "${WORKDIR}/multistrap.conf" || true diff --git
>> a/meta/conf/isar-bitbake.conf b/meta/conf/isar-bitbake.conf index
>> f85f5cc..5a26743 100644 --- a/meta/conf/isar-bitbake.conf
>> +++ b/meta/conf/isar-bitbake.conf
>> @@ -18,9 +18,10 @@
>>   # ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE
>> OR # OTHER DEALINGS IN THE SOFTWARE.
>>   
>> +WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PF}"
>>   DEPLOY_DIR_DEB = "${TMPDIR}/deploy/deb/${MACHINE}"
>>   SSTATE_DIR ?= "${TMPDIR}/sstate-cache"
>> -BUILDCHROOT_DIR =
>> "${TOPDIR}/tmp/work/buildchroot/${DISTRO}-${DISTRO_ARCH}/rootfs"
>> +BUILDCHROOT_DIR =
>> "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot/rootfs" # Setup
>> our default hash policy BB_SIGNATURE_HANDLER ?= "noop"
>> diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb
>> b/meta/recipes-devtools/buildchroot/buildchroot.bb index
>> ccba683..722007d 100644 ---
>> a/meta/recipes-devtools/buildchroot/buildchroot.bb +++
>> b/meta/recipes-devtools/buildchroot/buildchroot.bb @@ -21,7 +21,7 @@
>> BUILDCHROOT_PREINSTALL ?= "gcc \ apt \
>>                              automake"
>>   
>> -WORKDIR = "${TMPDIR}/work/${PF}/${DISTRO}"
>> +WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}"
>>   
>>   do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>>   
>> @@ -34,18 +34,20 @@ do_build() {
>>       install -m 755 ${THISDIR}/files/setup.sh ${WORKDIR}
>>       install -m 755 ${THISDIR}/files/download_dev-random
>> ${WORKDIR}/hooks_multistrap/
>> -    # Adjust multistrap config
>> -    sed -i 's|##BUILDCHROOT_PREINSTALL##|${BUILDCHROOT_PREINSTALL}|'
>> ${WORKDIR}/multistrap.conf
>> -    sed -i 's|##DISTRO##|${DISTRO}|' ${WORKDIR}/multistrap.conf
>> -    sed -i 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|'
>> ${WORKDIR}/multistrap.conf
>> -    sed -i 's|##DISTRO_SUITE##|${DISTRO_SUITE}|'
>> ${WORKDIR}/multistrap.conf
>> -    sed -i 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|'
>> ${WORKDIR}/multistrap.conf
>> -    sed -i
>> 's|##CONFIG_SCRIPT##|./tmp/work/${PF}/${DISTRO}/configscript.sh|'
>> ${WORKDIR}/multistrap.conf
>> -    sed -i 's|##SETUP_SCRIPT##|./tmp/work/${PF}/${DISTRO}/setup.sh|'
>> ${WORKDIR}/multistrap.conf
>> -    sed -i
>> 's|##DIR_HOOKS##|./tmp/work/${PF}/${DISTRO}/hooks_multistrap|'
>> ${WORKDIR}/multistrap.conf -
>> -    # Multistrap config use relative paths, so ensure that we are in
>> the right folder
>> +    # Multistrap accepts only relative path in configuration files,
>> so get it: cd ${TOPDIR}
>> +    WORKDIR_REL=$(python -c "import os.path; print
>> os.path.relpath('${WORKDIR}')") +
>> +    # Adjust multistrap config
>> +    sed -i -e 's|##IMAGE_PREINSTALL##|${IMAGE_PREINSTALL}|g' \
>> +           -e 's|##DISTRO##|${DISTRO}|g' \
>> +           -e 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|g' \
>> +           -e 's|##DISTRO_SUITE##|${DISTRO_SUITE}|g' \
>> +           -e 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|g' \
>> +           -e
>> 's|##CONFIG_SCRIPT##|./'"$WORKDIR_REL"'/configscript.sh|g' \
>> +           -e 's|##SETUP_SCRIPT##|./'"$WORKDIR_REL"'/setup.sh|g' \
>> +           -e
>> 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \
>> +              ${WORKDIR}/multistrap.conf
> 
> Arghh ... So easy on the eye and still wrong. Such style changes should
> be in another patch, but i guess i started it.
> 
> You remove ##BUILDCHROOT_PREINSTALL## and add ##IMAGE_PREINSTALL##.
> 

Already caught this issue during CI builds,fixed in my reply to this 
patch (I've resent this patch in-reply-to).

Alex

> Henning
> 
>>       # Create root filesystem
>>       sudo multistrap -a ${DISTRO_ARCH} -d "${BUILDCHROOT_DIR}" -f
>> "${WORKDIR}/multistrap.conf" || true
>

^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH v3 3/5] stamps: Name stamps canonicaly and unified
  2017-09-11 19:27 [PATCH v3 0/5] Unify work space for packages Alexander Smirnov
  2017-09-11 19:27 ` [PATCH v3 1/5] image: Unify path to image rootfs Alexander Smirnov
  2017-09-11 19:27 ` [PATCH v3 2/5] isar: Change build folders tree Alexander Smirnov
@ 2017-09-11 19:27 ` Alexander Smirnov
  2017-09-11 19:27 ` [PATCH v3 4/5] image: Name image " Alexander Smirnov
                   ` (2 subsequent siblings)
  5 siblings, 0 replies; 11+ messages in thread
From: Alexander Smirnov @ 2017-09-11 19:27 UTC (permalink / raw)
  To: isar-users; +Cc: Alexander Smirnov

Name task stamps canonically according to the build folders structure:
${DISTRO}-${DISTRO_ARCH} or ${DISTRO}-${MACHINE}.

Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
---
 meta/classes/ext4-img.bbclass | 2 +-
 meta/classes/image.bbclass    | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/ext4-img.bbclass b/meta/classes/ext4-img.bbclass
index f493037..9af781f 100644
--- a/meta/classes/ext4-img.bbclass
+++ b/meta/classes/ext4-img.bbclass
@@ -6,7 +6,7 @@ ROOTFS_EXTRA ?= "64"
 
 EXT4_IMAGE_FILE = "${DEPLOY_DIR_IMAGE}/${PN}-${MACHINE}-${DISTRO}.ext4.img"
 
-do_ext4_image[stamp-extra-info] = "${MACHINE}-${DISTRO}"
+do_ext4_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
 
 # Generate ext4 filesystem image
 do_ext4_image() {
diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index dff2705..590725c 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -10,7 +10,7 @@ IMAGE_ROOTFS   = "${WORKDIR}/rootfs"
 
 inherit ${IMAGE_TYPE}
 
-do_populate[stamp-extra-info] = "${MACHINE}-${DISTRO}"
+do_populate[stamp-extra-info] = "${DISTRO}-${MACHINE}"
 
 # Install Debian packages, that were built from sources
 do_populate() {
-- 
2.1.4


^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH v3 4/5] image: Name image canonicaly and unified
  2017-09-11 19:27 [PATCH v3 0/5] Unify work space for packages Alexander Smirnov
                   ` (2 preceding siblings ...)
  2017-09-11 19:27 ` [PATCH v3 3/5] stamps: Name stamps canonicaly and unified Alexander Smirnov
@ 2017-09-11 19:27 ` Alexander Smirnov
  2017-09-11 19:27 ` [PATCH v3 5/5] class/dpkg: Unify workplace for packages Alexander Smirnov
  2017-09-12 14:26 ` [PATCH] doc: Update technical overview Alexander Smirnov
  5 siblings, 0 replies; 11+ messages in thread
From: Alexander Smirnov @ 2017-09-11 19:27 UTC (permalink / raw)
  To: isar-users; +Cc: Alexander Smirnov

Name images according to the build folders structure:
${DISTRO}-${MACHINE}.

Now all the build artifacts have the same unified names.

Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
---
 meta/classes/ext4-img.bbclass | 2 +-
 scripts/start_vm              | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/meta/classes/ext4-img.bbclass b/meta/classes/ext4-img.bbclass
index 9af781f..3e3768b 100644
--- a/meta/classes/ext4-img.bbclass
+++ b/meta/classes/ext4-img.bbclass
@@ -4,7 +4,7 @@
 # Extra space for rootfs in MB
 ROOTFS_EXTRA ?= "64"
 
-EXT4_IMAGE_FILE = "${DEPLOY_DIR_IMAGE}/${PN}-${MACHINE}-${DISTRO}.ext4.img"
+EXT4_IMAGE_FILE = "${DEPLOY_DIR_IMAGE}/${PN}-${DISTRO}-${MACHINE}.ext4.img"
 
 do_ext4_image[stamp-extra-info] = "${DISTRO}-${MACHINE}"
 
diff --git a/scripts/start_vm b/scripts/start_vm
index 32d4864..1ce84b7 100755
--- a/scripts/start_vm
+++ b/scripts/start_vm
@@ -109,7 +109,7 @@ eval "$(egrep 'MACHINE_SERIAL' $MACHINE_CONF |bb2sh)"
 
 readonly CONFIG_CONF=$ISARROOT/meta-isar/conf/multiconfig/qemu$ARCH-$DISTRO.conf
 eval "$(egrep '(KERNEL|INITRD)_IMAGE|ROOTFS_DEV|QEMU_' $CONFIG_CONF |bb2sh)"
-readonly ROOTFS_IMAGE=isar-image-base-qemu$ARCH-debian-$DISTRO.ext4.img
+readonly ROOTFS_IMAGE=isar-image-base-debian-$DISTRO-qemu$ARCH.ext4.img
 
 QCPU=
 [ -n "$QEMU_CPU" ] && QCPU="-cpu $QEMU_CPU"
-- 
2.1.4


^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH v3 5/5] class/dpkg: Unify workplace for packages
  2017-09-11 19:27 [PATCH v3 0/5] Unify work space for packages Alexander Smirnov
                   ` (3 preceding siblings ...)
  2017-09-11 19:27 ` [PATCH v3 4/5] image: Name image " Alexander Smirnov
@ 2017-09-11 19:27 ` Alexander Smirnov
  2017-09-12 14:26 ` [PATCH] doc: Update technical overview Alexander Smirnov
  5 siblings, 0 replies; 11+ messages in thread
From: Alexander Smirnov @ 2017-09-11 19:27 UTC (permalink / raw)
  To: isar-users; +Cc: Alexander Smirnov

Get rid of splitting package's data between WORKDIR and BUILDCHROOT.
Now everything is stored into WORKDIR, and BUILDCHROOT is used for
compilation only.

Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
---
 meta/classes/dpkg.bbclass | 17 ++++++++++-------
 1 file changed, 10 insertions(+), 7 deletions(-)

diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
index 360a95c..16b7fc5 100644
--- a/meta/classes/dpkg.bbclass
+++ b/meta/classes/dpkg.bbclass
@@ -3,12 +3,11 @@
 
 # Add dependency from buildchroot creation
 DEPENDS += "buildchroot"
-do_unpack[deptask] = "do_build"
+do_build[deptask] = "do_build"
 
 # Each package should have its own unique build folder, so use
 # recipe name as identifier
 PP = "/home/builder/${PN}"
-BUILDROOT = "${BUILDCHROOT_DIR}/${PP}"
 
 do_fetch[dirs] = "${DL_DIR}"
 
@@ -27,17 +26,16 @@ python do_fetch() {
 
 addtask fetch before do_unpack
 
-do_unpack[dirs] = "${BUILDROOT}"
+do_unpack[dirs] = "${WORKDIR}"
 do_unpack[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
-S ?= "${BUILDROOT}"
 
-# Unpack package and put it into working directory in buildchroot
+# Unpack package and put it into working directory
 python do_unpack() {
     src_uri = (d.getVar('SRC_URI', True) or "").split()
     if len(src_uri) == 0:
         return
 
-    rootdir = d.getVar('BUILDROOT', True)
+    rootdir = d.getVar('WORKDIR', True)
 
     try:
         fetcher = bb.fetch2.Fetch(src_uri, d)
@@ -48,17 +46,22 @@ python do_unpack() {
 
 addtask unpack after do_fetch before do_build
 
+BUILDROOT = "${BUILDCHROOT_DIR}/${PP}"
 do_build[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
 
 # Build package from sources using build script
 do_build() {
+    mkdir -p ${BUILDROOT}
+    sudo mount --bind ${WORKDIR} ${BUILDROOT}
     sudo chroot ${BUILDCHROOT_DIR} /build.sh ${PP}/${SRC_DIR}
+    sudo umount ${BUILDROOT}
+    rm -rf ${BUILDROOT}
 }
 
 
 # Install package to dedicated deploy directory
 do_install() {
-    install -m 644 ${BUILDROOT}/*.deb ${DEPLOY_DIR_DEB}/
+    install -m 644 ${WORKDIR}/*.deb ${DEPLOY_DIR_DEB}/
 }
 
 addtask install after do_build
-- 
2.1.4


^ permalink raw reply	[flat|nested] 11+ messages in thread

* [PATCH] doc: Update technical overview
  2017-09-11 19:27 [PATCH v3 0/5] Unify work space for packages Alexander Smirnov
                   ` (4 preceding siblings ...)
  2017-09-11 19:27 ` [PATCH v3 5/5] class/dpkg: Unify workplace for packages Alexander Smirnov
@ 2017-09-12 14:26 ` Alexander Smirnov
  5 siblings, 0 replies; 11+ messages in thread
From: Alexander Smirnov @ 2017-09-12 14:26 UTC (permalink / raw)
  To: isar-users; +Cc: Alexander Smirnov

BUILDCHROOT is no more used for storing sources to build custom package.
This patch updates the respective topic in technical overview.

Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
---
 doc/technical_overview.md | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/doc/technical_overview.md b/doc/technical_overview.md
index a4563b3..0c86577 100644
--- a/doc/technical_overview.md
+++ b/doc/technical_overview.md
@@ -160,11 +160,11 @@ contain debian folder. The build process is implemented in
 
 1. Task `do_fetch`: fetch source code from external link
 
-2. Task `do_unpack`: unpack source code to
-   `${BUILDCHROOT_DIR}/home/build/${PN}`
+2. Task `do_unpack`: unpack source code to `${WORKDIR}`
 
-3. Task `do_build`: switch to buildchroot using chroot command and run
-   `build.sh` script. The `build.sh` script performs the following:
+3. Task `do_build`: mount folder with source code to buildchroot, switch
+   to buildchroot using chroot command and run `build.sh` script. The
+   `build.sh` script performs the following:
 
    1. Go to `/home/build/${PN}`
 
@@ -173,8 +173,7 @@ contain debian folder. The build process is implemented in
    3. Run dpkg-buildpackage
 
 4. Task `do_install`: install successfully built packages
-   `${BUILDCHROOT_DIR}/home/build/${PN}/*.deb` to deploy directory
-   `${DEPLOY_DIR_DEB}`
+   `${WORKDIR}/*.deb` to deploy directory `${DEPLOY_DIR_DEB}`
 
 ## 3.5 Populate Target Filesystem
 
-- 
2.1.4


^ permalink raw reply	[flat|nested] 11+ messages in thread

end of thread, other threads:[~2017-09-12 14:26 UTC | newest]

Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-09-11 19:27 [PATCH v3 0/5] Unify work space for packages Alexander Smirnov
2017-09-11 19:27 ` [PATCH v3 1/5] image: Unify path to image rootfs Alexander Smirnov
2017-09-11 19:27 ` [PATCH v3 2/5] isar: Change build folders tree Alexander Smirnov
2017-09-12  7:23   ` Claudius Heine
2017-09-12  7:57   ` Alexander Smirnov
2017-09-12 12:52   ` Henning Schild
2017-09-12 13:02     ` Alexander Smirnov
2017-09-11 19:27 ` [PATCH v3 3/5] stamps: Name stamps canonicaly and unified Alexander Smirnov
2017-09-11 19:27 ` [PATCH v3 4/5] image: Name image " Alexander Smirnov
2017-09-11 19:27 ` [PATCH v3 5/5] class/dpkg: Unify workplace for packages Alexander Smirnov
2017-09-12 14:26 ` [PATCH] doc: Update technical overview Alexander Smirnov

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox