[PATCH 0/3] Update Isar documentation

[PATCH 0/3] Update Isar documentation

[Anton Mikanovich]

Update Isar documentation files.

Anton Mikanovich (3):
  mount: Add unmount requirement to API changelog
  doc: Update tasks description
  doc: Update available image types

 RECIPE-API-CHANGELOG.md   | 21 +++++++++++++++
 doc/technical_overview.md | 56 +++++++++++++++++++++++----------------
 doc/user_manual.md        | 10 +++++--
 3 files changed, 62 insertions(+), 25 deletions(-)

-- 
2.25.1

[PATCH 1/3] mount: Add unmount requirement to API changelog

[Anton Mikanovich]

After the latest mount rebuild any usage of mount functions should be
ended with the following unmount call.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 RECIPE-API-CHANGELOG.md | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 806c291..546d91c 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -292,3 +292,24 @@ Migrate your patches so they can be applied with "git am", or
 
 Kernel image name for arm64 platforms is vmlinux now. Image format was
 not changed (uncompressed executable) but now it is named correctly.
+
+### Make unmounting mandatory
+
+There were mount API to provide some path inside chroots:
+```
+dpkg_do_mounts
+buildchroot_do_mounts
+image_do_mounts
+rootfs_do_mounts
+```
+
+From now those calls should be always followed by unmount actions:
+
+```
+dpkg_undo_mounts
+buildchroot_undo_mounts
+image_undo_mounts
+rootfs_undo_mounts
+```
+
+It case any unmounted paths left at the end of build warnings will be thrown.
-- 
2.25.1

[PATCH 2/3] doc: Update tasks description

[Anton Mikanovich]

Update task list and description for `dpkg.bbclass` and
`dpkg-raw.bbclass` to correspond with the latest state.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 doc/technical_overview.md | 56 +++++++++++++++++++++++----------------
 1 file changed, 33 insertions(+), 23 deletions(-)

diff --git a/doc/technical_overview.md b/doc/technical_overview.md
index bbd22b6..a0e7ac8 100644
--- a/doc/technical_overview.md
+++ b/doc/technical_overview.md
@@ -214,41 +214,51 @@ Both consist of the following steps:
 
 2. Task `do_unpack`: unpack those files to `${WORKDIR}`
 
-3. Task `do_install` _only_ for `dpkg-raw`: copy all you want in your
+3. Task `do_transform_template`: process `${TEMPLATE_FILES}` to substitute
+   variables listed in `${TEMPLATE_VARS}` in template files
+
+4. Task `do_apt_fetch`: fetch source files from apt repos
+
+5. Task `do_apt_unpack`: unpack those files to `${WORKDIR}`
+
+6. Task `do_patch`: process .patch files in the `${SRC_URI}`
+
+7. Task `do_install` _only_ for `dpkg-raw`: copy all you want in your
    debian package to `${D}`, install hooks in `${D}/DEBIAN`
 
-4. Task `do_prepare': perform any preparation steps to the unpacked/patched
-   sources before the build. This task calls the dpkg_prepare shell function
-   with the buildchroot mounts in place (`dpkg_do_mounts')
+8. Task `do_adjust_git`: adjust alternates links inside git repos in the
+   package workdir to make them valid inside chroot
+
+9. Task `do_prepare_build': perform any preparation steps to the unpacked and
+   patched sources before the build.
 
-   4.1. the `dpkg_prepare` function of `dpkg.bbclass` runs `/isar/deps.sh` in
-        the buildchroot. That performs the following:
+   For `dpkg-raw` it perform `deb_debianize` call to translate the recipe
+   meta-data into a debian/* files suitable for building with dpkg-buildpackage
 
-        1. Go to `/home/build/${PN}`
+10. Task `do_install_builddeps`: install build dependencies of the package.
+    This task calls the `builddeps_install` shell function with the buildchroot
+    mounts in place (surrounded by `dpkg_do_mounts` and `dpkg_undo_mounts`).
 
-        2. Get list of dependencies from debian/control and install them
+    The `builddeps_install` function of `dpkg.bbclass` runs `/isar/deps.sh` in
+    the buildchroot. That performs the following:
 
-   4.2. the `dpkg_prepare` function of `dpkg-raw.bbclass` translate the
-        recipe meta-data into a debian/control file suitable for packaging
-        with dpkg-deb
+    10.1. Go to `/home/build/${PN}`
 
-5. Task `do_build`: mount folder (`dpkg_do_mounts') with unpacked files to buildchroot,
-   execute the actual build function `dpkg_runbuild`, and finally umount again
-   (`dpkg_undo_mounts')
+    10.2. Get list of dependencies from debian/control and install them
 
-   5.1. the `dpkg_runbuild` function of `dpkg.bbclass` runs `build.sh` in the
-        buildchroot. That performs the following:
+11. Task `do_dpkg_build`: mount folder (`dpkg_do_mounts`) with unpacked files
+    to buildchroot, execute the actual build function `dpkg_runbuild`, and
+    finally umount again (`dpkg_undo_mounts`)
 
-        1. Go to `/home/build/${PN}`
+    The `dpkg_runbuild` function of `dpkg.bbclass` runs `/isar/build.sh` in
+    the buildchroot. That performs the following:
 
-        2. Run dpkg-buildpackage
+    11.1. Go to `/home/build/${PN}`
 
-   5.2. the `dpkg_runbuild` function of `dpkg-raw.bbclass` basically runs
-        `dpkg-deb` to construct a Debian package from a folder of files,
-	without compiling anything
+    11.2. Run dpkg-buildpackage
 
-6. Task `do_deploy_deb`: install successfully built packages
-   `${WORKDIR}/*.deb` to deploy directory `${DEPLOY_DIR_DEB}`
+12. Task `do_deploy_deb`: install successfully built packages
+    `${WORKDIR}/*.deb` to deploy directory `${DEPLOY_DIR_DEB}`
 
 ## 3.6 Populate Target Filesystem
 
-- 
2.25.1

[PATCH 3/3] doc: Update available image types

[Anton Mikanovich]

Update the list of image types can be used for targets.

Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
---
 doc/user_manual.md | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/doc/user_manual.md b/doc/user_manual.md
index cf7dc2f..ab8dd03 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -450,10 +450,14 @@ Please refer to `Add a Custom Application` section for more information about wr
 
 Isar can generate various images types for specific machine. The type of the image to be generated may be specified through the `IMAGE_TYPE` variable. Currently, the following image types are provided:
 
- - `ext4` - Raw ext4 filesystem image (default option for `qemuarm` machine).
+ - `ext4-img` - Raw ext4 filesystem image (default option for `qemuarm` machine).
  - `rpi-sdimg` - A complete, partitioned Raspberry Pi SD card image (default option for the `rpi` machine).
  - `wic-img` - A full disk image with user-specified partitions created and populated using the wic tool.
- - `ubi-img` - A image for use on mtd nand partitions employing UBI
+ - `targz-img` - Compressed Tar archive with filesystem.
+ - `fit-img` - Flattened Image Tree filesystem image with user-specified binaries included.
+ - `ubifs-img` - UBIFS filesystem image to be used with ubiupdatevol.
+ - `ubi-img` - An image for use on mtd nand partitions employing UBI.
+ - `container-img` - Container image specified by `CONTAINER_FORMAT` value.
 
 ---
 
@@ -581,9 +585,11 @@ Isar contains additional image type classes that can be used as reference:
  - `ext4-img`
  - `rpi-sdimg`
  - `targz-img`
+ - `fit-img`
  - `ubifs-img`
  - `ubi-img`
  - `wic-img`
+ - `container-img`
 
 ---
 
-- 
2.25.1

Re: [PATCH 1/3] mount: Add unmount requirement to API changelog

[Jan Kiszka]

On 30.06.21 13:35, Anton Mikanovich wrote:
> After the latest mount rebuild any usage of mount functions should be
> ended with the following unmount call.
> 
> Signed-off-by: Anton Mikanovich <amikan@ilbers.de>
> ---
>  RECIPE-API-CHANGELOG.md | 21 +++++++++++++++++++++
>  1 file changed, 21 insertions(+)
> 
> diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
> index 806c291..546d91c 100644
> --- a/RECIPE-API-CHANGELOG.md
> +++ b/RECIPE-API-CHANGELOG.md
> @@ -292,3 +292,24 @@ Migrate your patches so they can be applied with "git am", or
>  
>  Kernel image name for arm64 platforms is vmlinux now. Image format was
>  not changed (uncompressed executable) but now it is named correctly.
> +
> +### Make unmounting mandatory
> +
> +There were mount API to provide some path inside chroots:
> +```
> +dpkg_do_mounts
> +buildchroot_do_mounts
> +image_do_mounts
> +rootfs_do_mounts
> +```
> +
> +From now those calls should be always followed by unmount actions:
> +
> +```
> +dpkg_undo_mounts
> +buildchroot_undo_mounts
> +image_undo_mounts
> +rootfs_undo_mounts
> +```
> +
> +It case any unmounted paths left at the end of build warnings will be thrown.
> 

That's only one part of the story:

dpkg_do_mounts, e.g., used to be a shell function. Now it's a python
function. So,

do_my_task() {
    dpkg_do_mounts
    sudo chroot ...
    dpkg_undo_mounts
}

like in [1] no longer works and requires to factor the shell part and
rewrite the main task in python.

BTW, [1] already followed that pattern, so it's not really a new thing.

Jan

[1] https://github.com/siemens/meta-iot2050/blob/master/classes/npm.bbclass

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux