[PATCH 0/4] Fixes/improvements to local apt repo to cache upstream debian packages for offline usage

[PATCH 0/4] Fixes/improvements to local apt repo to cache upstream debian packages for offline usage

[Maxim Yu. Osipov]

Hello,

This series addresses fixes/suggestions by Henning Schild 
to v2 series of "local apt repo to cache upstream debian packages for offline usage"

Regards,
Maxim.

Maxim Yu. Osipov (4):
  isar-bootstrap: eliminate workaround --allow-unauthenticated
  isar-bootdtrap-helper: test explicit value
  base-apt-helper: fix missed echo in command
  base-apt-helper: Introduce function compare_pkg_md5sums

 meta/classes/base-apt-helper.bbclass               | 24 ++++++++++++----------
 meta/classes/isar-bootstrap-helper.bbclass         |  4 ++--
 .../isar-bootstrap/files/base-apt-sources          |  2 +-
 meta/recipes-devtools/buildchroot/files/deps.sh    |  2 +-
 4 files changed, 17 insertions(+), 15 deletions(-)

-- 
2.11.0

[PATCH 1/4] isar-bootstrap: eliminate workaround --allow-unauthenticated

[Maxim Yu. Osipov]

use [trusted=yes] in base-apt-sources instead.

Suggested-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Maxim Yu. Osipov <mosipov@ilbers.de>
---
 meta/classes/isar-bootstrap-helper.bbclass              | 2 +-
 meta/recipes-core/isar-bootstrap/files/base-apt-sources | 2 +-
 meta/recipes-devtools/buildchroot/files/deps.sh         | 2 +-
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/meta/classes/isar-bootstrap-helper.bbclass b/meta/classes/isar-bootstrap-helper.bbclass
index 24e1157..8f5e583 100644
--- a/meta/classes/isar-bootstrap-helper.bbclass
+++ b/meta/classes/isar-bootstrap-helper.bbclass
@@ -79,7 +79,7 @@ setup_root_file_system() {
     ROOTFSDIR="$1"
     shift
     PACKAGES="$@"
-    APT_ARGS="install --yes --allow-unauthenticated -o Debug::pkgProblemResolver=yes"
+    APT_ARGS="install --yes -o Debug::pkgProblemResolver=yes"
     CLEAN_FILES="${ROOTFSDIR}/etc/hostname ${ROOTFSDIR}/etc/resolv.conf"
 
     sudo cp -Trpfx \
diff --git a/meta/recipes-core/isar-bootstrap/files/base-apt-sources b/meta/recipes-core/isar-bootstrap/files/base-apt-sources
index 594db56..e806465 100644
--- a/meta/recipes-core/isar-bootstrap/files/base-apt-sources
+++ b/meta/recipes-core/isar-bootstrap/files/base-apt-sources
@@ -1 +1 @@
-deb file:///base-apt/debian {DISTRO} main
+deb [trusted=yes] file:///base-apt/debian {DISTRO} main
diff --git a/meta/recipes-devtools/buildchroot/files/deps.sh b/meta/recipes-devtools/buildchroot/files/deps.sh
index 2fc2f66..4bd604f 100644
--- a/meta/recipes-devtools/buildchroot/files/deps.sh
+++ b/meta/recipes-devtools/buildchroot/files/deps.sh
@@ -10,7 +10,7 @@ source /isar/common.sh
 # Notes:
 #   1) everything before the -y switch is unchanged from the defaults
 #   2) we add -y to go non-interactive
-install_cmd="apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends -y --allow-unauthenticated"
+install_cmd="apt-get -o Debug::pkgProblemResolver=yes --no-install-recommends -y"
 
 # Make sure that we have latest isar-apt content.
 # Options meaning:
-- 
2.11.0

[PATCH 2/4] isar-bootdtrap-helper: test explicit value

[Maxim Yu. Osipov]

Suggested-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Maxim Yu. Osipov <mosipov@ilbers.de>
---
 meta/classes/isar-bootstrap-helper.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/isar-bootstrap-helper.bbclass b/meta/classes/isar-bootstrap-helper.bbclass
index 8f5e583..a316fe8 100644
--- a/meta/classes/isar-bootstrap-helper.bbclass
+++ b/meta/classes/isar-bootstrap-helper.bbclass
@@ -137,7 +137,7 @@ setup_root_file_system() {
             /usr/bin/apt-get purge --yes $pkg
     done
     if [ ${CLEAN} ]; then
-        if [ ${KEEP_APT_CACHE} ]; then
+        if [ ${KEEP_APT_CACHE} -eq 1 ]; then
             mkdir -p ${WORKDIR}/apt_cache
             sudo mv $(find $ROOTFSDIR/var/cache/apt -name '*.deb') ${WORKDIR}/apt_cache
             sudo chown $USER ${WORKDIR}/apt_cache/*
-- 
2.11.0

[PATCH 3/4] base-apt-helper: fix missed echo in command

[Maxim Yu. Osipov]

Signed-off-by: Maxim Yu. Osipov <mosipov@ilbers.de>
---
 meta/classes/base-apt-helper.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/base-apt-helper.bbclass b/meta/classes/base-apt-helper.bbclass
index ff4a3d5..7c58b7c 100644
--- a/meta/classes/base-apt-helper.bbclass
+++ b/meta/classes/base-apt-helper.bbclass
@@ -37,7 +37,7 @@ populate_base_apt() {
             fi
 
             # md5sum differs, so remove the package from base-apt
-            name=$($base_name | cut -d '_' -f 1)
+            name=$(echo $base_name | cut -d '_' -f 1)
             reprepro -b ${REPO_BASE_DIR}/${DISTRO_NAME} \
                      --dbdir ${REPO_BASE_DB_DIR}/${DISTRO_NAME} \
                      -C main -A ${DISTRO_ARCH} \
-- 
2.11.0

[PATCH 4/4] base-apt-helper: Introduce function compare_pkg_md5sums

[Maxim Yu. Osipov]

Reuse common code block as a function.

Suggested-by: Henning Schild <henning.schild@siemens.com>
Signed-off-by: Maxim Yu. Osipov <mosipov@ilbers.de>
---
 meta/classes/base-apt-helper.bbclass | 22 ++++++++++++----------
 1 file changed, 12 insertions(+), 10 deletions(-)

diff --git a/meta/classes/base-apt-helper.bbclass b/meta/classes/base-apt-helper.bbclass
index 7c58b7c..eb28f76 100644
--- a/meta/classes/base-apt-helper.bbclass
+++ b/meta/classes/base-apt-helper.bbclass
@@ -4,6 +4,16 @@
 DISTRO_NAME ?= "${@ d.getVar('DISTRO', True).split('-')[0]}"
 DISTRO_SUITE ?= "${@ d.getVar('DISTRO', True).split('-')[1]}"
 
+compare_pkg_md5sums() {
+   pkg1=$1
+   pkg2=$2
+
+   md1=$(md5sum $pkg1 | cut -d ' ' -f 1)
+   md2=$(md5sum $pkg2 | cut -d ' ' -f 1)
+
+   [ "$md1" = "$md2" ]
+}
+
 populate_base_apt() {
     search_dir=$1
 
@@ -20,21 +30,13 @@ populate_base_apt() {
         if [ -n "$isar_package" ]; then
             # Check if MD5 sums are identical. This helps to avoid the case
             # when packages is overridden from another repo.
-            md1=$(md5sum $package | cut -d ' ' -f 1)
-            md2=$(md5sum $isar_package | cut -d ' ' -f 1)
-            if [ "$md1" = "$md2" ]; then
-                continue
-            fi
+            compare_pkg_md5sums "$package" "$isar_package" && continue
         fi
 
         # Check if this package is already in base-apt
         isar_package=$(find ${REPO_BASE_DIR}/${DISTRO_NAME} -name $base_name)
         if [ -n "$isar_package" ]; then
-            md1=$(md5sum $package | cut -d ' ' -f 1)
-            md2=$(md5sum $isar_package | cut -d ' ' -f 1)
-            if [ "$md1" = "$md2" ]; then
-                continue
-            fi
+            compare_pkg_md5sums "$package" "$isar_package" && continue
 
             # md5sum differs, so remove the package from base-apt
             name=$(echo $base_name | cut -d '_' -f 1)
-- 
2.11.0

Re: [PATCH 0/4] Fixes/improvements to local apt repo to cache upstream debian packages for offline usage

[Henning Schild]

Hi,

looks good. But the chown $USER problem is still in there, which breaks
the build for me and probably all others using the kas/docker workflow.

Henning

Am Fri, 26 Oct 2018 12:32:37 +0200
schrieb "Maxim Yu. Osipov" <mosipov@ilbers.de>:

> Hello,
> 
> This series addresses fixes/suggestions by Henning Schild 
> to v2 series of "local apt repo to cache upstream debian packages for
> offline usage"
> 
> Regards,
> Maxim.
> 
> Maxim Yu. Osipov (4):
>   isar-bootstrap: eliminate workaround --allow-unauthenticated
>   isar-bootdtrap-helper: test explicit value
>   base-apt-helper: fix missed echo in command
>   base-apt-helper: Introduce function compare_pkg_md5sums
> 
>  meta/classes/base-apt-helper.bbclass               | 24
> ++++++++++++----------
> meta/classes/isar-bootstrap-helper.bbclass         |  4
> ++-- .../isar-bootstrap/files/base-apt-sources          |  2 +-
> meta/recipes-devtools/buildchroot/files/deps.sh    |  2 +- 4 files
> changed, 17 insertions(+), 15 deletions(-)
> 

Re: [PATCH 0/4] Fixes/improvements to local apt repo to cache upstream debian packages for offline usage

[Maxim Yu. Osipov]

On 10/26/18 1:32 PM, Maxim Yu. Osipov wrote:
> Hello,
> 
> This series addresses fixes/suggestions by Henning Schild
> to v2 series of "local apt repo to cache upstream debian packages for offline usage"

Applied to the 'next',

Maxim.

> Regards,
> Maxim.
> 
> Maxim Yu. Osipov (4):
>    isar-bootstrap: eliminate workaround --allow-unauthenticated
>    isar-bootdtrap-helper: test explicit value
>    base-apt-helper: fix missed echo in command
>    base-apt-helper: Introduce function compare_pkg_md5sums
> 
>   meta/classes/base-apt-helper.bbclass               | 24 ++++++++++++----------
>   meta/classes/isar-bootstrap-helper.bbclass         |  4 ++--
>   .../isar-bootstrap/files/base-apt-sources          |  2 +-
>   meta/recipes-devtools/buildchroot/files/deps.sh    |  2 +-
>   4 files changed, 17 insertions(+), 15 deletions(-)
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov