[PATCH 0/5] support creation of a full repo for offline/reproducible builds

[PATCH 0/5] support creation of a full repo for offline/reproducible builds

[Cedric_Hombourger]

From: Cedric Hombourger <Cedric_Hombourger@mentor.com>

The package repository created by isar using reprepro only includes packages by isar.
To support offline/reproducible builds, this changeset adds a do_populate task to
augment the repo with packages used during the build. The task may be used against
the buildchroot and images recipes. It should be noted that isar currently assumes
that the base distribution will provide both an -updates and security feed. This is
certainly true for Debian but may not be the case for other distributions or when
when using our own feed.

Some rework may be needed if the isar-apt changes get merged first. Conceptually the
implementation may not change much (as far as I can tell!)

Please review and let me know if any rework is required.

Cedric Hombourger (5):
  base: add populate_repo task to include distro packages to the repo
  meta: move reprepro handling code to its own class
  buildchroot: use reprepro to populate the full repo
  reprepro: create the -updates distribution
  multistrap: make the security feed optional

 .../recipes-core/images/files/distributions.in     |  4 +
 .../recipes-core/images/files/multistrap.conf.in   | 14 ++--
 meta-isar/recipes-core/images/isar-image-base.bb   | 86 ++++++++++++++++++++++
 meta/classes/image.bbclass                         | 47 +-----------
 meta/classes/reprepro.bbclass                      | 52 +++++++++++++
 meta/recipes-devtools/buildchroot/buildchroot.bb   | 20 +++++
 .../buildchroot/files/distributions.in             |  7 ++
 .../buildchroot/files/multistrap.conf.in           | 10 +--
 8 files changed, 182 insertions(+), 58 deletions(-)
 create mode 100644 meta/classes/reprepro.bbclass
 create mode 100644 meta/recipes-devtools/buildchroot/files/distributions.in

-- 
2.11.0

[PATCH 1/5] base: add populate_repo task to include distro packages to the repo

[Cedric_Hombourger]

From: Cedric Hombourger <Cedric_Hombourger@mentor.com>

A package repository is already created for custom packages (built by
Isar). Provide a populate_repo task to also include any packages
included into images built by Isar. This mechanism will add both binary
and source packages. The augmented repository may be used later for
reproducible builds (technically, packages pulled for the buildchroots
we build should also be added; this will be the subject of another
changeset).

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 .../recipes-core/images/files/multistrap.conf.in   |  6 +-
 meta-isar/recipes-core/images/isar-image-base.bb   | 78 ++++++++++++++++++++++
 2 files changed, 81 insertions(+), 3 deletions(-)

diff --git a/meta-isar/recipes-core/images/files/multistrap.conf.in b/meta-isar/recipes-core/images/files/multistrap.conf.in
index 432b6af..8e4e276 100644
--- a/meta-isar/recipes-core/images/files/multistrap.conf.in
+++ b/meta-isar/recipes-core/images/files/multistrap.conf.in
@@ -16,19 +16,19 @@ source=##DISTRO_APT_SOURCE##
 suite=##DISTRO_SUITE##
 components=##DISTRO_COMPONENTS##
 packages=##IMAGE_PREINSTALL##
-omitdebsrc=true
+omitdebsrc=false
 
 [updates]
 source=##DISTRO_APT_SOURCE##
 suite=##DISTRO_SUITE##-updates
 components=##DISTRO_COMPONENTS##
-omitdebsrc=true
+omitdebsrc=false
 
 [security]
 source=##DISTRO_APT_SOURCE_SEC##
 suite=##DISTRO_SUITE##/updates
 components=##DISTRO_COMPONENTS##
-omitdebsrc=true
+omitdebsrc=false
 
 [Isar]
 packages=##IMAGE_INSTALL##
diff --git a/meta-isar/recipes-core/images/isar-image-base.bb b/meta-isar/recipes-core/images/isar-image-base.bb
index dea2cdb..6ce3d6d 100644
--- a/meta-isar/recipes-core/images/isar-image-base.bb
+++ b/meta-isar/recipes-core/images/isar-image-base.bb
@@ -52,6 +52,79 @@ do_rootfs_populate_dev() {
     _mknod_cdev 666 ptmx    5 2
 }
 
+do_populate_repo() {
+
+    APT_CACHEDIR=${IMAGE_ROOTFS}/var/cache/apt # Home of the apt cache (for the target image)
+    APT_DEBDIR=${APT_CACHEDIR}/archives        # Downloaded packages can be found there
+    APT_SRCDIR=${WORKDIR}/apt-sources          # Put source packages outside of the rootfs
+
+    sudo mkdir -p ${APT_SRCDIR}
+
+    apt_status_file=$(mktemp)
+    for p in ${DEPLOY_DIR_DEB}/${PN}/*.deb; do
+        # Only process packages that were NOT built by Isar (already added to the repo)
+        deb=$(basename ${p})
+        if [ ! -f ${DEPLOY_DIR_DEB}/${deb} ]; then
+            # Extract package name
+            pkg=$(dpkg-deb -I ${p}|grep -e '^ Package:'|sed -e 's,^ Package: ,,g')
+
+            # Extract package section and priority so we can tell rerepro where to place this package
+            sect=$(dpkg-deb -I ${p}|grep -e '^ Section:'|sed -e 's,^ Section: ,,g')
+            prio=$(dpkg-deb -I ${p}|grep -e '^ Priority:'|sed -e 's,^ Priority: ,,g')
+
+            # Determine name of the source package (same as the binary package if not specified)
+            src=$(dpkg-deb -I ${p}|grep -e '^ Source:'|sed -e 's,^ Source: ,,g'|awk '{ print $1; }')
+            src=${src:-${pkg}}
+
+            # Add the binary package to the repo
+            call_reprepro -b ${DEPLOY_DIR_APT}/${DISTRO}   \
+                --dbdir ${DEPLOY_DIR_DB}/${DISTRO}         \
+                -C main -S ${sect} -P ${prio}              \
+                includedeb ${DEBDISTRONAME} ${p}
+
+            # Get source package and add it to the repo
+            # Notes:
+            #   - multistrap --source-dir is failing to fetch sources hence this manual process
+            #     (the culprit appears to be the following apt-get option: APT::Default-Release='*')
+            #   - the apt-get command-line was copied from multistrap and tweaked for sources
+            #     (a separate/temporary apt status file is used to avoid polluting the existing one
+            #      with source transactions)
+            #   - as we assume sources were already fetched if the directory exists, quietly
+            #     attempt to delete it first; if sources are indeed there, it will fetch. If
+            #     they aren't, the directory will be deleted and we will have another go at it
+            rmdir ${APT_SRCDIR}/${src} 2>/dev/null || true
+            if [ ! -d "${APT_SRCDIR}/${src}" ]; then
+                sudo mkdir -p ${APT_SRCDIR}/${src}
+                cd ${APT_SRCDIR}/${src}
+                bbnote "fetching source package '${src}' (binary: ${pkg})"
+                sudo -E apt-get \
+                    -o Dir::Etc::TrustedParts=${IMAGE_ROOTFS}/etc/apt/trusted.gpg.d \
+                    -o Dir::Etc::Trusted=${IMAGE_ROOTFS}/etc/apt/trusted.gpg \
+                    -o Apt::Get::AllowUnauthenticated=true \
+                    -o Apt::Get::Download-Only=true \
+                    -o Apt::Install-Recommends=false \
+                    -o Dir=${IMAGE_ROOTFS} \
+                    -o Dir::Etc=${IMAGE_ROOTFS}/etc/apt/ \
+                    -o Dir::Etc::Parts=${IMAGE_ROOTFS}/etc/apt/apt.conf.d/ \
+                    -o Dir::Etc::PreferencesParts=${IMAGE_ROOTFS}/etc/apt/preferences.d/ \
+                    -o Dir::State=${IMAGE_ROOTFS}/var/lib/apt/ \
+                    -o Dir::State::Status=${apt_status_file} \
+                    -o Dir::Cache=${IMAGE_ROOTFS}/var/cache/apt/ -y source ${src}
+                call_reprepro -b ${DEPLOY_DIR_APT}/${DISTRO}   \
+                    --dbdir ${DEPLOY_DIR_DB}/${DISTRO}         \
+                    -C main -S ${sect} -P ${prio}              \
+                    includedsc ${DEBDISTRONAME}                \
+                    ${APT_SRCDIR}/${src}/*.dsc
+            else
+                bbnote "already fetched source package '${src}' (binary: ${pkg})"
+            fi
+        fi
+    done
+    rm -f ${apt_status_file}
+}
+
+addtask populate_repo
+
 do_rootfs() {
     E="${@ bb.utils.export_proxies(d)}"
 
@@ -94,6 +167,11 @@ do_rootfs() {
     # because of proxy settings
     sudo -E multistrap -a ${DISTRO_ARCH} -d "${IMAGE_ROOTFS}" -f "${WORKDIR}/multistrap.conf"
 
+    # Copy packages before they vanish when the root file-systems get configured
+    # (apt purge is being run)
+    install -m 755 -d ${DEPLOY_DIR_DEB}/${PN}
+    sudo cp -l ${IMAGE_ROOTFS}/var/cache/apt/archives/*.deb ${DEPLOY_DIR_DEB}/${PN}/
+
     # Configure root filesystem
     sudo chroot ${IMAGE_ROOTFS} /${DISTRO_CONFIG_SCRIPT} ${MACHINE_SERIAL} ${BAUDRATE_TTY} \
         ${ROOTFS_DEV}
-- 
2.11.0

Re: [PATCH 1/5] base: add populate_repo task to include distro packages to the repo

[Alexander Smirnov]

Hi,

n 02/04/2018 08:54 PM, Cedric_Hombourger@mentor.com wrote:
> From: Cedric Hombourger <Cedric_Hombourger@mentor.com>
> 
> A package repository is already created for custom packages (built by
> Isar). Provide a populate_repo task to also include any packages
> included into images built by Isar. This mechanism will add both binary
> and source packages. The augmented repository may be used later for
> reproducible builds (technically, packages pulled for the buildchroots
> we build should also be added; this will be the subject of another
> changeset).

Isar-apt should never be used to mirror upstream repositories. Isar-apt 
intended to be an overlay for pure Debian.

Regarding reproducibility, there are already two series discusses in 
this list which introduce dedicated base-apt repository for caching 
upstream:

https://groups.google.com/d/msg/isar-users/9b0f6LP4X6k/LJ3mLxE1EAAJ
https://groups.google.com/d/msg/isar-users/QQUsVmSaAGk/B6FGnVWhAQAJ

Alex

> 
> Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
> ---
>   .../recipes-core/images/files/multistrap.conf.in   |  6 +-
>   meta-isar/recipes-core/images/isar-image-base.bb   | 78 ++++++++++++++++++++++
>   2 files changed, 81 insertions(+), 3 deletions(-)
> 
> diff --git a/meta-isar/recipes-core/images/files/multistrap.conf.in b/meta-isar/recipes-core/images/files/multistrap.conf.in
> index 432b6af..8e4e276 100644
> --- a/meta-isar/recipes-core/images/files/multistrap.conf.in
> +++ b/meta-isar/recipes-core/images/files/multistrap.conf.in
> @@ -16,19 +16,19 @@ source=##DISTRO_APT_SOURCE##
>   suite=##DISTRO_SUITE##
>   components=##DISTRO_COMPONENTS##
>   packages=##IMAGE_PREINSTALL##
> -omitdebsrc=true
> +omitdebsrc=false
>   
>   [updates]
>   source=##DISTRO_APT_SOURCE##
>   suite=##DISTRO_SUITE##-updates
>   components=##DISTRO_COMPONENTS##
> -omitdebsrc=true
> +omitdebsrc=false
>   
>   [security]
>   source=##DISTRO_APT_SOURCE_SEC##
>   suite=##DISTRO_SUITE##/updates
>   components=##DISTRO_COMPONENTS##
> -omitdebsrc=true
> +omitdebsrc=false
>   
>   [Isar]
>   packages=##IMAGE_INSTALL##
> diff --git a/meta-isar/recipes-core/images/isar-image-base.bb b/meta-isar/recipes-core/images/isar-image-base.bb
> index dea2cdb..6ce3d6d 100644
> --- a/meta-isar/recipes-core/images/isar-image-base.bb
> +++ b/meta-isar/recipes-core/images/isar-image-base.bb
> @@ -52,6 +52,79 @@ do_rootfs_populate_dev() {
>       _mknod_cdev 666 ptmx    5 2
>   }
>   
> +do_populate_repo() {
> +
> +    APT_CACHEDIR=${IMAGE_ROOTFS}/var/cache/apt # Home of the apt cache (for the target image)
> +    APT_DEBDIR=${APT_CACHEDIR}/archives        # Downloaded packages can be found there
> +    APT_SRCDIR=${WORKDIR}/apt-sources          # Put source packages outside of the rootfs
> +
> +    sudo mkdir -p ${APT_SRCDIR}
> +
> +    apt_status_file=$(mktemp)
> +    for p in ${DEPLOY_DIR_DEB}/${PN}/*.deb; do
> +        # Only process packages that were NOT built by Isar (already added to the repo)
> +        deb=$(basename ${p})
> +        if [ ! -f ${DEPLOY_DIR_DEB}/${deb} ]; then
> +            # Extract package name
> +            pkg=$(dpkg-deb -I ${p}|grep -e '^ Package:'|sed -e 's,^ Package: ,,g')
> +
> +            # Extract package section and priority so we can tell rerepro where to place this package
> +            sect=$(dpkg-deb -I ${p}|grep -e '^ Section:'|sed -e 's,^ Section: ,,g')
> +            prio=$(dpkg-deb -I ${p}|grep -e '^ Priority:'|sed -e 's,^ Priority: ,,g')
> +
> +            # Determine name of the source package (same as the binary package if not specified)
> +            src=$(dpkg-deb -I ${p}|grep -e '^ Source:'|sed -e 's,^ Source: ,,g'|awk '{ print $1; }')
> +            src=${src:-${pkg}}
> +
> +            # Add the binary package to the repo
> +            call_reprepro -b ${DEPLOY_DIR_APT}/${DISTRO}   \
> +                --dbdir ${DEPLOY_DIR_DB}/${DISTRO}         \
> +                -C main -S ${sect} -P ${prio}              \
> +                includedeb ${DEBDISTRONAME} ${p}
> +
> +            # Get source package and add it to the repo
> +            # Notes:
> +            #   - multistrap --source-dir is failing to fetch sources hence this manual process
> +            #     (the culprit appears to be the following apt-get option: APT::Default-Release='*')
> +            #   - the apt-get command-line was copied from multistrap and tweaked for sources
> +            #     (a separate/temporary apt status file is used to avoid polluting the existing one
> +            #      with source transactions)
> +            #   - as we assume sources were already fetched if the directory exists, quietly
> +            #     attempt to delete it first; if sources are indeed there, it will fetch. If
> +            #     they aren't, the directory will be deleted and we will have another go at it
> +            rmdir ${APT_SRCDIR}/${src} 2>/dev/null || true
> +            if [ ! -d "${APT_SRCDIR}/${src}" ]; then
> +                sudo mkdir -p ${APT_SRCDIR}/${src}
> +                cd ${APT_SRCDIR}/${src}
> +                bbnote "fetching source package '${src}' (binary: ${pkg})"
> +                sudo -E apt-get \
> +                    -o Dir::Etc::TrustedParts=${IMAGE_ROOTFS}/etc/apt/trusted.gpg.d \
> +                    -o Dir::Etc::Trusted=${IMAGE_ROOTFS}/etc/apt/trusted.gpg \
> +                    -o Apt::Get::AllowUnauthenticated=true \
> +                    -o Apt::Get::Download-Only=true \
> +                    -o Apt::Install-Recommends=false \
> +                    -o Dir=${IMAGE_ROOTFS} \
> +                    -o Dir::Etc=${IMAGE_ROOTFS}/etc/apt/ \
> +                    -o Dir::Etc::Parts=${IMAGE_ROOTFS}/etc/apt/apt.conf.d/ \
> +                    -o Dir::Etc::PreferencesParts=${IMAGE_ROOTFS}/etc/apt/preferences.d/ \
> +                    -o Dir::State=${IMAGE_ROOTFS}/var/lib/apt/ \
> +                    -o Dir::State::Status=${apt_status_file} \
> +                    -o Dir::Cache=${IMAGE_ROOTFS}/var/cache/apt/ -y source ${src}
> +                call_reprepro -b ${DEPLOY_DIR_APT}/${DISTRO}   \
> +                    --dbdir ${DEPLOY_DIR_DB}/${DISTRO}         \
> +                    -C main -S ${sect} -P ${prio}              \
> +                    includedsc ${DEBDISTRONAME}                \
> +                    ${APT_SRCDIR}/${src}/*.dsc
> +            else
> +                bbnote "already fetched source package '${src}' (binary: ${pkg})"
> +            fi
> +        fi
> +    done
> +    rm -f ${apt_status_file}
> +}
> +
> +addtask populate_repo
> +
>   do_rootfs() {
>       E="${@ bb.utils.export_proxies(d)}"
>   
> @@ -94,6 +167,11 @@ do_rootfs() {
>       # because of proxy settings
>       sudo -E multistrap -a ${DISTRO_ARCH} -d "${IMAGE_ROOTFS}" -f "${WORKDIR}/multistrap.conf"
>   
> +    # Copy packages before they vanish when the root file-systems get configured
> +    # (apt purge is being run)
> +    install -m 755 -d ${DEPLOY_DIR_DEB}/${PN}
> +    sudo cp -l ${IMAGE_ROOTFS}/var/cache/apt/archives/*.deb ${DEPLOY_DIR_DEB}/${PN}/
> +
>       # Configure root filesystem
>       sudo chroot ${IMAGE_ROOTFS} /${DISTRO_CONFIG_SCRIPT} ${MACHINE_SERIAL} ${BAUDRATE_TTY} \
>           ${ROOTFS_DEV}
> 

-- 
With best regards,
Alexander Smirnov

ilbers GmbH
Baierbrunner Str. 28c
D-81379 Munich
+49 (89) 122 67 24-0
http://ilbers.de/
Commercial register Munich, HRB 214197
General manager: Baurzhan Ismagulov

[PATCH 2/5] meta: move reprepro handling code to its own class

[Cedric_Hombourger]

From: Cedric Hombourger <Cedric_Hombourger@mentor.com>

The reprepro handling code (cache creation and update) may be
re-used in the buildchroot recipe to later add required build
packages to the full repo (if one should be created). Move the
code to its own class and inherit it from the image class (the
only user at this time).

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/classes/image.bbclass    | 47 +----------------------------------------
 meta/classes/reprepro.bbclass | 49 +++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 50 insertions(+), 46 deletions(-)
 create mode 100644 meta/classes/reprepro.bbclass

diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
index e2cb01b..b20db18 100644
--- a/meta/classes/image.bbclass
+++ b/meta/classes/image.bbclass
@@ -17,53 +17,8 @@ KERNEL_IMAGE ?= "${@get_image_name(d, 'vmlinuz')}"
 INITRD_IMAGE ?= "${@get_image_name(d, 'initrd.img')}"
 
 inherit ${IMAGE_TYPE}
-call_reprepro() {
-    for i in $(seq 1 ${REPREPRO_LOCK_ATTEMPTS}); do
-        #  According to `sh` manual page, shell exit statuses fall between
-        # 0-255. The EEXIST error code is (-17), so casting to usigned 8-bit
-        # integer results value (239).
-        eexist=$(python -c 'import errno; print(256-errno.EEXIST)')
-        retval="0"
-        reprepro $* || retval="$?"
-
-        # If reprepro has failed to get database lock, it returns EEXIST code.
-        # In this case we continue trying to get lock until max amount of
-        # attempts is reached.
-        if [ $retval -eq $eexist ]; then
-            bbwarn "Failed to get reprepro lock, trying again..."
-            sleep 5
-        else
-            break
-        fi
-    done
-
-    if [ $retval -ne 0 ]; then
-        bbfatal "reprepro failed"
-    fi
-}
-
-CACHE_CONF_DIR = "${DEPLOY_DIR_APT}/${DISTRO}/conf"
-do_cache_config[dirs] = "${CACHE_CONF_DIR}"
-do_cache_config[stamp-extra-info] = "${DISTRO}"
-
-# Generate reprepro config for current distro if it doesn't exist. Once it's
-# generated, this task should do nothing.
-do_cache_config() {
-    if [ ! -e "${CACHE_CONF_DIR}/distributions" ]; then
-        sed -e "s#{DISTRO_NAME}#"${DEBDISTRONAME}"#g" \
-            ${FILESDIR}/distributions.in > ${CACHE_CONF_DIR}/distributions
-    fi
-
-    path_cache="${DEPLOY_DIR_APT}/${DISTRO}"
-    path_databases="${DEPLOY_DIR_DB}/${DISTRO}"
-
-    if [ ! -d "${path_databases}" ]; then
-        call_reprepro -b ${path_cache} \
-                      --dbdir ${path_databases} \
-                      export ${DEBDISTRONAME}
-    fi
-}
 
+inherit reprepro
 addtask cache_config before do_populate
 
 do_populate[stamp-extra-info] = "${DISTRO}-${MACHINE}"
diff --git a/meta/classes/reprepro.bbclass b/meta/classes/reprepro.bbclass
new file mode 100644
index 0000000..dfe48cb
--- /dev/null
+++ b/meta/classes/reprepro.bbclass
@@ -0,0 +1,49 @@
+# This software is a part of ISAR.
+# Copyright (C) 2015-2017 ilbers GmbH
+
+CACHE_CONF_DIR = "${DEPLOY_DIR_APT}/${DISTRO}/conf"
+do_cache_config[dirs] = "${CACHE_CONF_DIR}"
+do_cache_config[stamp-extra-info] = "${DISTRO}"
+
+# Generate reprepro config for current distro if it doesn't exist. Once it's
+# generated, this task should do nothing.
+do_cache_config() {
+    if [ ! -e "${CACHE_CONF_DIR}/distributions" ]; then
+        sed -e "s#{DISTRO_NAME}#"${DEBDISTRONAME}"#g" \
+            ${FILESDIR}/distributions.in > ${CACHE_CONF_DIR}/distributions
+    fi
+
+    path_cache="${DEPLOY_DIR_APT}/${DISTRO}"
+    path_databases="${DEPLOY_DIR_DB}/${DISTRO}"
+
+    if [ ! -d "${path_databases}" ]; then
+        call_reprepro -b ${path_cache} \
+                      --dbdir ${path_databases} \
+                      export ${DEBDISTRONAME}
+    fi
+}
+
+call_reprepro() {
+    for i in $(seq 1 ${REPREPRO_LOCK_ATTEMPTS}); do
+        #  According to `sh` manual page, shell exit statuses fall between
+        # 0-255. The EEXIST error code is (-17), so casting to usigned 8-bit
+        # integer results value (239).
+        eexist=$(python -c 'import errno; print(256-errno.EEXIST)')
+        retval="0"
+        reprepro $* || retval="$?"
+
+        # If reprepro has failed to get database lock, it returns EEXIST code.
+        # In this case we continue trying to get lock until max amount of
+        # attempts is reached.
+        if [ $retval -eq $eexist ]; then
+            bbwarn "Failed to get reprepro lock, trying again..."
+            sleep 5
+        else
+            break
+        fi
+    done
+
+    if [ $retval -ne 0 ]; then
+        bbfatal "reprepro failed"
+    fi
+}
-- 
2.11.0

[PATCH 3/5] buildchroot: use reprepro to populate the full repo

[Cedric_Hombourger]

From: Cedric Hombourger <Cedric_Hombourger@mentor.com>

Add the populate_repo task to the buildchroot so its packages may
also be included in the full repo.

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta/recipes-devtools/buildchroot/buildchroot.bb         | 12 ++++++++++++
 meta/recipes-devtools/buildchroot/files/distributions.in |  3 +++
 2 files changed, 15 insertions(+)
 create mode 100644 meta/recipes-devtools/buildchroot/files/distributions.in

diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb
index 3bae081..4ebd090 100644
--- a/meta/recipes-devtools/buildchroot/buildchroot.bb
+++ b/meta/recipes-devtools/buildchroot/buildchroot.bb
@@ -3,6 +3,9 @@
 # This software is a part of ISAR.
 # Copyright (C) 2015-2016 ilbers GmbH
 
+inherit reprepro
+addtask cache_config before do_build
+
 DESCRIPTION = "Multistrap development filesystem"
 
 LICENSE = "gpl-2.0"
@@ -88,3 +91,12 @@ do_build() {
     sudo chroot ${BUILDCHROOT_DIR} /configscript.sh
     _do_build_cleanup
 }
+
+do_populate_repo() {
+    call_reprepro -b ${DEPLOY_DIR_APT}/${DISTRO}        \
+        --dbdir ${DEPLOY_DIR_DB}/${DISTRO}              \
+        -C main includedeb ${DEBDISTRONAME}             \
+        ${BUILDCHROOT_DIR}/var/cache/apt/archives/*.deb
+}
+
+addtask populate_repo
diff --git a/meta/recipes-devtools/buildchroot/files/distributions.in b/meta/recipes-devtools/buildchroot/files/distributions.in
new file mode 100644
index 0000000..cd214c6
--- /dev/null
+++ b/meta/recipes-devtools/buildchroot/files/distributions.in
@@ -0,0 +1,3 @@
+Codename: {DISTRO_NAME}
+Architectures: i386 armhf amd64 source
+Components: main
-- 
2.11.0

[PATCH 4/5] reprepro: create the -updates distribution

[Cedric_Hombourger]

From: Cedric Hombourger <Cedric_Hombourger@mentor.com>

To allow use of the full repo we create, the -updates distribution
should be created since expected by the multistrap configurations
we create.

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta-isar/recipes-core/images/files/distributions.in     | 4 ++++
 meta/classes/reprepro.bbclass                            | 3 +++
 meta/recipes-devtools/buildchroot/files/distributions.in | 4 ++++
 3 files changed, 11 insertions(+)

diff --git a/meta-isar/recipes-core/images/files/distributions.in b/meta-isar/recipes-core/images/files/distributions.in
index cd214c6..1c6bb51 100644
--- a/meta-isar/recipes-core/images/files/distributions.in
+++ b/meta-isar/recipes-core/images/files/distributions.in
@@ -1,3 +1,7 @@
 Codename: {DISTRO_NAME}
 Architectures: i386 armhf amd64 source
 Components: main
+
+Codename: {DISTRO_NAME}-updates
+Architectures: i386 armhf amd64 source
+Components: main
diff --git a/meta/classes/reprepro.bbclass b/meta/classes/reprepro.bbclass
index dfe48cb..52ead45 100644
--- a/meta/classes/reprepro.bbclass
+++ b/meta/classes/reprepro.bbclass
@@ -20,6 +20,9 @@ do_cache_config() {
         call_reprepro -b ${path_cache} \
                       --dbdir ${path_databases} \
                       export ${DEBDISTRONAME}
+        call_reprepro -b ${path_cache} \
+                      --dbdir ${path_databases} \
+                      export ${DEBDISTRONAME}-updates
     fi
 }
 
diff --git a/meta/recipes-devtools/buildchroot/files/distributions.in b/meta/recipes-devtools/buildchroot/files/distributions.in
index cd214c6..1c6bb51 100644
--- a/meta/recipes-devtools/buildchroot/files/distributions.in
+++ b/meta/recipes-devtools/buildchroot/files/distributions.in
@@ -1,3 +1,7 @@
 Codename: {DISTRO_NAME}
 Architectures: i386 armhf amd64 source
 Components: main
+
+Codename: {DISTRO_NAME}-updates
+Architectures: i386 armhf amd64 source
+Components: main
-- 
2.11.0

[PATCH 5/5] multistrap: make the security feed optional

[Cedric_Hombourger]

From: Cedric Hombourger <Cedric_Hombourger@mentor.com>

Leave the security feed commented in the generated multistrap
configuration if DISTRO_APT_SOURCE_SEC is empty or set to "none"

Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
---
 meta-isar/recipes-core/images/files/multistrap.conf.in     | 10 +++++-----
 meta-isar/recipes-core/images/isar-image-base.bb           |  8 ++++++++
 meta/recipes-devtools/buildchroot/buildchroot.bb           |  8 ++++++++
 meta/recipes-devtools/buildchroot/files/multistrap.conf.in | 10 +++++-----
 4 files changed, 26 insertions(+), 10 deletions(-)

diff --git a/meta-isar/recipes-core/images/files/multistrap.conf.in b/meta-isar/recipes-core/images/files/multistrap.conf.in
index 8e4e276..94db7fa 100644
--- a/meta-isar/recipes-core/images/files/multistrap.conf.in
+++ b/meta-isar/recipes-core/images/files/multistrap.conf.in
@@ -24,11 +24,11 @@ suite=##DISTRO_SUITE##-updates
 components=##DISTRO_COMPONENTS##
 omitdebsrc=false
 
-[security]
-source=##DISTRO_APT_SOURCE_SEC##
-suite=##DISTRO_SUITE##/updates
-components=##DISTRO_COMPONENTS##
-omitdebsrc=false
+##SEC_ENABLE_DISABLE## [security]
+##SEC_ENABLE_DISABLE## source=##DISTRO_APT_SOURCE_SEC##
+##SEC_ENABLE_DISABLE## suite=##DISTRO_SUITE##/updates
+##SEC_ENABLE_DISABLE## components=##DISTRO_COMPONENTS##
+##SEC_ENABLE_DISABLE## omitdebsrc=true
 
 [Isar]
 packages=##IMAGE_INSTALL##
diff --git a/meta-isar/recipes-core/images/isar-image-base.bb b/meta-isar/recipes-core/images/isar-image-base.bb
index 6ce3d6d..a2c2282 100644
--- a/meta-isar/recipes-core/images/isar-image-base.bb
+++ b/meta-isar/recipes-core/images/isar-image-base.bb
@@ -135,12 +135,20 @@ do_rootfs() {
     cd ${TOPDIR}
     WORKDIR_REL=${@ os.path.relpath(d.getVar("WORKDIR", True))}
 
+    # Check whether to include the distro security feed in the multistrap config
+    if [ -z "${DISTRO_APT_SOURCE_SEC}" ] || [ "${DISTRO_APT_SOURCE_SEC}" = "none" ]; then
+        sec_enable_disable="#"
+    else
+        sec_enable_disable=""
+    fi
+
     # Adjust multistrap config
     sed -e 's|##IMAGE_PREINSTALL##|${IMAGE_PREINSTALL}|g' \
         -e 's|##DISTRO_MULTICONF_BOOTSTRAP##|${DISTRO_MULTICONF_BOOTSTRAP}|g' \
         -e 's|##DISTRO_MULTICONF_APTSOURCES##|${DISTRO_MULTICONF_APTSOURCES}|g' \
         -e 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|g' \
         -e 's|##DISTRO_APT_SOURCE_SEC##|${DISTRO_APT_SOURCE_SEC}|g' \
+        -e 's|##SEC_ENABLE_DISABLE##|'"$sec_enable_disable"'|g' \
         -e 's|##DISTRO_SUITE##|${DISTRO_SUITE}|g' \
         -e 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|g' \
         -e 's|##CONFIG_SCRIPT##|./'"$WORKDIR_REL"'/${DISTRO_CONFIG_SCRIPT}|g' \
diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb
index 4ebd090..fa335b3 100644
--- a/meta/recipes-devtools/buildchroot/buildchroot.bb
+++ b/meta/recipes-devtools/buildchroot/buildchroot.bb
@@ -47,12 +47,20 @@ do_build() {
     cd ${TOPDIR}
     WORKDIR_REL=${@ os.path.relpath(d.getVar("WORKDIR", True))}
 
+    # Check whether to include the distro security feed in the multistrap config
+    if [ -z "${DISTRO_APT_SOURCE_SEC}" ] || [ "${DISTRO_APT_SOURCE_SEC}" = "none" ]; then
+        sec_enable_disable="#"
+    else
+        sec_enable_disable=""
+    fi
+
     # Adjust multistrap config
     sed -e 's|##BUILDCHROOT_PREINSTALL##|${BUILDCHROOT_PREINSTALL}|g' \
         -e 's|##DISTRO_MULTICONF_BOOTSTRAP##|${DISTRO_MULTICONF_BOOTSTRAP}|g' \
         -e 's|##DISTRO_MULTICONF_APTSOURCES##|${DISTRO_MULTICONF_APTSOURCES}|g' \
         -e 's|##DISTRO_APT_SOURCE##|${DISTRO_APT_SOURCE}|g' \
         -e 's|##DISTRO_APT_SOURCE_SEC##|${DISTRO_APT_SOURCE_SEC}|g' \
+        -e 's|##SEC_ENABLE_DISABLE##|'"$sec_enable_disable"'|g' \
         -e 's|##DISTRO_SUITE##|${DISTRO_SUITE}|g' \
         -e 's|##DISTRO_COMPONENTS##|${DISTRO_COMPONENTS}|g' \
         -e 's|##CONFIG_SCRIPT##|./'"$WORKDIR_REL"'/configscript.sh|g' \
diff --git a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in b/meta/recipes-devtools/buildchroot/files/multistrap.conf.in
index a0b28e3..1e3425f 100644
--- a/meta/recipes-devtools/buildchroot/files/multistrap.conf.in
+++ b/meta/recipes-devtools/buildchroot/files/multistrap.conf.in
@@ -24,8 +24,8 @@ suite=##DISTRO_SUITE##-updates
 components=##DISTRO_COMPONENTS##
 omitdebsrc=true
 
-[security]
-source=##DISTRO_APT_SOURCE_SEC##
-suite=##DISTRO_SUITE##/updates
-components=##DISTRO_COMPONENTS##
-omitdebsrc=true
+##SEC_ENABLE_DISABLE## [security]
+##SEC_ENABLE_DISABLE## source=##DISTRO_APT_SOURCE_SEC##
+##SEC_ENABLE_DISABLE## suite=##DISTRO_SUITE##/updates
+##SEC_ENABLE_DISABLE## components=##DISTRO_COMPONENTS##
+##SEC_ENABLE_DISABLE## omitdebsrc=true
-- 
2.11.0

Re: [PATCH 0/5] support creation of a full repo for offline/reproducible builds

[Jan Kiszka]

On 2018-02-04 18:54, Cedric_Hombourger@mentor.com wrote:
> From: Cedric Hombourger <Cedric_Hombourger@mentor.com>
> 
> The package repository created by isar using reprepro only includes packages by isar.
> To support offline/reproducible builds, this changeset adds a do_populate task to
> augment the repo with packages used during the build. The task may be used against
> the buildchroot and images recipes. It should be noted that isar currently assumes
> that the base distribution will provide both an -updates and security feed. This is
> certainly true for Debian but may not be the case for other distributions or when
> when using our own feed.

The automatic addition of update and security feeds is more of a
workaround until we have multi-repo support like Claudius is working on.
I guess we can then drop this and just have repo lists for the different
Debian versions with multiple entries.

> 
> Some rework may be needed if the isar-apt changes get merged first. Conceptually the
> implementation may not change much (as far as I can tell!)
> 
> Please review and let me know if any rework is required.

There is indeed quite some overlap with what Alex and I were discussing
yesterday at FOSDEM. However, also looking at these patches, we need to
do some homework first. As you correctly stated in patch 1, there is the
issue that we pull package list twice at different times: first for the
buildchroot and then again for the image. That needs to go first so that
we end up with a consistent build. Also, all those duplications in logic
between the two chroot setup recipes are killing us.

So I would propose the following roadmap:

- consolidate chroot building into a common class that both buildchroot
  and image recipes derive from, while doing that
   - generate multistrap.conf (e.g. "cat <<EOF") instead of doing sed
     all over the place
- build up a single apt cache that all chroot builders use
- derive the mirror repo list after the first installation from that
  cache which will then contain ALL required packages in the right
  versions
- describe the workflow (doc/) how to generate that mirror and how to
  use it in succeeding builds

Makes sense?

We should than clarify who will work on what.

BTW, the very first step is sorting out all the other patches and
applying them to next. For that, Alex plans to first update bitbake and
then go through what is pending (and still merges fine).

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH 0/5] support creation of a full repo for offline/reproducible builds

[Benedikt Niedermayr]

Am 05.02.2018 um 08:18 schrieb Jan Kiszka:
> On 2018-02-04 18:54, Cedric_Hombourger@mentor.com wrote:
>> From: Cedric Hombourger <Cedric_Hombourger@mentor.com>
>>
>> The package repository created by isar using reprepro only includes packages by isar.
>> To support offline/reproducible builds, this changeset adds a do_populate task to
>> augment the repo with packages used during the build. The task may be used against
>> the buildchroot and images recipes. It should be noted that isar currently assumes
>> that the base distribution will provide both an -updates and security feed. This is
>> certainly true for Debian but may not be the case for other distributions or when
>> when using our own feed.
> The automatic addition of update and security feeds is more of a
> workaround until we have multi-repo support like Claudius is working on.
> I guess we can then drop this and just have repo lists for the different
> Debian versions with multiple entries.
>
>> Some rework may be needed if the isar-apt changes get merged first. Conceptually the
>> implementation may not change much (as far as I can tell!)
>>
>> Please review and let me know if any rework is required.
> There is indeed quite some overlap with what Alex and I were discussing
> yesterday at FOSDEM. However, also looking at these patches, we need to
> do some homework first. As you correctly stated in patch 1, there is the
> issue that we pull package list twice at different times: first for the
> buildchroot and then again for the image. That needs to go first so that
> we end up with a consistent build. Also, all those duplications in logic
> between the two chroot setup recipes are killing us.
>
> So I would propose the following roadmap:
>
> - consolidate chroot building into a common class that both buildchroot
>    and image recipes derive from, while doing that
>     - generate multistrap.conf (e.g. "cat <<EOF") instead of doing sed
>       all over the place
> - build up a single apt cache that all chroot builders use
> - derive the mirror repo list after the first installation from that
>    cache which will then contain ALL required packages in the right
>    versions
> - describe the workflow (doc/) how to generate that mirror and how to
>    use it in succeeding builds
>
> Makes sense?
>
> We should than clarify who will work on what.
>
> BTW, the very first step is sorting out all the other patches and
> applying them to next. For that, Alex plans to first update bitbake and
> then go through what is pending (and still merges fine).
>
> Jan
>

Hi,

> - consolidate chroot building into a common class that both buildchroot
>    and image recipes derive from, while doing that
Yes, sounds good.

> - generate multistrap.conf (e.g. "cat <<EOF") instead of doing sed
>       all over the place
Maybe creating a new task like do_multistrap_conf() written in Python 
may help here. Python could import the "configparser" module

for creating the config file? AFAIK the configparser module is included 
in standart python installation, so no further host system dependencies.

> - build up a single apt cache that all chroot builders use
> - describe the workflow (doc/) how to generate that mirror and how to
>    use it in succeeding builds

So we will separate building the apt cache from the Isar build process 
itself? Means, the user will setup the Mirror manually?

> - derive the mirror repo list after the first installation from that
>    cache which will then contain ALL required packages in the right
>    versions
Can you explain that in more detail?


Benni

Re: [PATCH 0/5] support creation of a full repo for offline/reproducible builds

[Alexander Smirnov]

On 02/05/2018 10:18 AM, Jan Kiszka wrote:
> On 2018-02-04 18:54, Cedric_Hombourger@mentor.com wrote:
>> From: Cedric Hombourger <Cedric_Hombourger@mentor.com>
>>
>> The package repository created by isar using reprepro only includes packages by isar.
>> To support offline/reproducible builds, this changeset adds a do_populate task to
>> augment the repo with packages used during the build. The task may be used against
>> the buildchroot and images recipes. It should be noted that isar currently assumes
>> that the base distribution will provide both an -updates and security feed. This is
>> certainly true for Debian but may not be the case for other distributions or when
>> when using our own feed.
> 
> The automatic addition of update and security feeds is more of a
> workaround until we have multi-repo support like Claudius is working on.
> I guess we can then drop this and just have repo lists for the different
> Debian versions with multiple entries.
> 
>>
>> Some rework may be needed if the isar-apt changes get merged first. Conceptually the
>> implementation may not change much (as far as I can tell!)
>>
>> Please review and let me know if any rework is required.
> 
> There is indeed quite some overlap with what Alex and I were discussing
> yesterday at FOSDEM. However, also looking at these patches, we need to
> do some homework first. As you correctly stated in patch 1, there is the
> issue that we pull package list twice at different times: first for the
> buildchroot and then again for the image. That needs to go first so that
> we end up with a consistent build. Also, all those duplications in logic
> between the two chroot setup recipes are killing us.
> 
> So I would propose the following roadmap:
> 
> - consolidate chroot building into a common class that both buildchroot
>    and image recipes derive from, while doing that
>     - generate multistrap.conf (e.g. "cat <<EOF") instead of doing sed
>       all over the place

I'd propose here to start with buildchroot/image recipes clean up. I 
mean there are lots of files in:

  - meta/recipes-devtools/buildchroot/files
  - meta-isar/recipes-core/images/files

Until these files will be unified, it makes no sense to implement common 
class - you anyway have to keep several sets of these files per 
image/buildchroot. That's why I proposed some time ago to drop 
multistrap hooks and implement their actions inside bitbake recipe, but 
this idea wasn't found support here.

> - build up a single apt cache that all chroot builders use

In other words, there should be the only one component/recipe who 
fetches the upstream packages.

There is still open question, how to fetch all the packages: in a single 
step or during the build.

I want to have an option similar to Yocto's one: -c fetchall to have 
possibility to fetch all the necessary artifacts and do not depend from 
network anymore.

> - derive the mirror repo list after the first installation from that
>    cache which will then contain ALL required packages in the right
>    versions
> - describe the workflow (doc/) how to generate that mirror and how to
>    use it in succeeding builds
> 
> Makes sense?
> 
> We should than clarify who will work on what.
> 
> BTW, the very first step is sorting out all the other patches and
> applying them to next. For that, Alex plans to first update bitbake and
> then go through what is pending (and still merges fine).

My wish will be to keep an eye on this list and synchronize your 
activities to avoid overlaps.

Alex

Re: [PATCH 0/5] support creation of a full repo for offline/reproducible builds

[Jan Kiszka]

On 2018-02-05 11:26, Alexander Smirnov wrote:
> On 02/05/2018 10:18 AM, Jan Kiszka wrote:
>> On 2018-02-04 18:54, Cedric_Hombourger@mentor.com wrote:
>>> From: Cedric Hombourger <Cedric_Hombourger@mentor.com>
>>>
>>> The package repository created by isar using reprepro only includes
>>> packages by isar.
>>> To support offline/reproducible builds, this changeset adds a
>>> do_populate task to
>>> augment the repo with packages used during the build. The task may be
>>> used against
>>> the buildchroot and images recipes. It should be noted that isar
>>> currently assumes
>>> that the base distribution will provide both an -updates and security
>>> feed. This is
>>> certainly true for Debian but may not be the case for other
>>> distributions or when
>>> when using our own feed.
>>
>> The automatic addition of update and security feeds is more of a
>> workaround until we have multi-repo support like Claudius is working on.
>> I guess we can then drop this and just have repo lists for the different
>> Debian versions with multiple entries.
>>
>>>
>>> Some rework may be needed if the isar-apt changes get merged first.
>>> Conceptually the
>>> implementation may not change much (as far as I can tell!)
>>>
>>> Please review and let me know if any rework is required.
>>
>> There is indeed quite some overlap with what Alex and I were discussing
>> yesterday at FOSDEM. However, also looking at these patches, we need to
>> do some homework first. As you correctly stated in patch 1, there is the
>> issue that we pull package list twice at different times: first for the
>> buildchroot and then again for the image. That needs to go first so that
>> we end up with a consistent build. Also, all those duplications in logic
>> between the two chroot setup recipes are killing us.
>>
>> So I would propose the following roadmap:
>>
>> - consolidate chroot building into a common class that both buildchroot
>>    and image recipes derive from, while doing that
>>     - generate multistrap.conf (e.g. "cat <<EOF") instead of doing sed
>>       all over the place
> 
> I'd propose here to start with buildchroot/image recipes clean up. I
> mean there are lots of files in:
> 
>  - meta/recipes-devtools/buildchroot/files
>  - meta-isar/recipes-core/images/files
> 
> Until these files will be unified, it makes no sense to implement common
> class - you anyway have to keep several sets of these files per
> image/buildchroot. That's why I proposed some time ago to drop
> multistrap hooks and implement their actions inside bitbake recipe, but
> this idea wasn't found support here.

Did you start that effort back then? I don't remember if there was some
code already or just the idea.

In any case, the similarity is too high to keep on with the two
instances. Let's start this now.

> 
>> - build up a single apt cache that all chroot builders use
> 
> In other words, there should be the only one component/recipe who
> fetches the upstream packages.
> 
> There is still open question, how to fetch all the packages: in a single
> step or during the build.

I think - as long as we fetch the package list only once - we can get
away for now with fetching the actual deb files when they are needed.

> 
> I want to have an option similar to Yocto's one: -c fetchall to have
> possibility to fetch all the necessary artifacts and do not depend from
> network anymore.

Let's look into this as an add-on feature later, I would say. I you see
that our implementation is making that needlessly harder, raise your
hand, though.

> 
>> - derive the mirror repo list after the first installation from that
>>    cache which will then contain ALL required packages in the right
>>    versions
>> - describe the workflow (doc/) how to generate that mirror and how to
>>    use it in succeeding builds
>>
>> Makes sense?
>>
>> We should than clarify who will work on what.
>>
>> BTW, the very first step is sorting out all the other patches and
>> applying them to next. For that, Alex plans to first update bitbake and
>> then go through what is pending (and still merges fine).
> 
> My wish will be to keep an eye on this list and synchronize your
> activities to avoid overlaps.
> 

Right, mine as well.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH 0/5] support creation of a full repo for offline/reproducible builds

[Benedikt Niedermayr]

Am 05.02.2018 um 11:26 schrieb Alexander Smirnov:
> In other words, there should be the only one component/recipe who 
> fetches the upstream packages.
>
> There is still open question, how to fetch all the packages: in a 
> single step or during the build.
>
> I want to have an option similar to Yocto's one: -c fetchall to have 
> possibility to fetch all the necessary artifacts and do not depend 
> from network anymore. 

Meanwhile I'm with you, in terms of your proposal. Fetching all packages 
at the beginning, would be easier to achive.
Otherwise I have no Idea how to solve potential remote repository 
updates between buildchroot and rootfs steps.


The other problem would be to resolve dependencies at the beginning of 
the build process.
I think "aptly" can do that hard work for us.
But this would require to replace reprepro with aptly.

I will implement a PoC with aptly which should cover the following:

1. Extract DEBIAN_DEPENDS from all recipes, as well as IMAGE_PREINSTALL 
and BUILDCHROOT_PREINSTALL.

2. Create an aptly mirror by setting the "-filter" option.
Aptly will resolve all dependencies when setting the filter option.
This is the most important step. If aptly fails to resolve dependencies 
correctly, I will abort my PoC.

3. Create a local Repo for Isar packages.

4. Start the build and let do_rootfs() refer to local mirror.


Benni

Re: [PATCH 0/5] support creation of a full repo for offline/reproducible builds

[Benedikt Niedermayr]

Am 04.02.2018 um 18:54 schrieb Cedric_Hombourger@mentor.com:
> From: Cedric Hombourger <Cedric_Hombourger@mentor.com>
>
> The package repository created by isar using reprepro only includes packages by isar.
> To support offline/reproducible builds, this changeset adds a do_populate task to
> augment the repo with packages used during the build. The task may be used against
> the buildchroot and images recipes. It should be noted that isar currently assumes
> that the base distribution will provide both an -updates and security feed. This is
> certainly true for Debian but may not be the case for other distributions or when
> when using our own feed.
>
> Some rework may be needed if the isar-apt changes get merged first. Conceptually the
> implementation may not change much (as far as I can tell!)
>
> Please review and let me know if any rework is required.
>
> Cedric Hombourger (5):
>    base: add populate_repo task to include distro packages to the repo
>    meta: move reprepro handling code to its own class
>    buildchroot: use reprepro to populate the full repo
>    reprepro: create the -updates distribution
>    multistrap: make the security feed optional
>
>   .../recipes-core/images/files/distributions.in     |  4 +
>   .../recipes-core/images/files/multistrap.conf.in   | 14 ++--
>   meta-isar/recipes-core/images/isar-image-base.bb   | 86 ++++++++++++++++++++++
>   meta/classes/image.bbclass                         | 47 +-----------
>   meta/classes/reprepro.bbclass                      | 52 +++++++++++++
>   meta/recipes-devtools/buildchroot/buildchroot.bb   | 20 +++++
>   .../buildchroot/files/distributions.in             |  7 ++
>   .../buildchroot/files/multistrap.conf.in           | 10 +--
>   8 files changed, 182 insertions(+), 58 deletions(-)
>   create mode 100644 meta/classes/reprepro.bbclass
>   create mode 100644 meta/recipes-devtools/buildchroot/files/distributions.in
>
Hi Cedric,

thanks.

Pulling packages from the local repository for subsequent builds is not 
implemented here?


Benni

Re: [PATCH 0/5] support creation of a full repo for offline/reproducible builds

[chombourger]

[-- Attachment #1.1: Type: text/plain, Size: 560 bytes --]

Hi Benni,

I did not have anything to do to make subsequent builds pull from the local 
repo other than publishing that repo and amending my environment as follows:

export DISTRO_APT_SOURCE="http://192.168.30.11/apt/foobar"
export DISTRO_APT_SOURCE_SEC="none"
export DISTRO_SUITE="foobar"
export DISTRO_COMPONENTS="main"
unset http_proxy # I typically use apt-cacher-ng, not needed when pulling 
local feeds

Cedric


Hi Cedric, 
>
> thanks. 
>
> Pulling packages from the local repository for subsequent builds is not 
> implemented here? 
>
>
> Benni 
>
>
>

[-- Attachment #1.2: Type: text/html, Size: 812 bytes --]