public inbox for isar-users@googlegroups.com
 help / color / mirror / Atom feed
From: "Maxim Yu. Osipov" <mosipov@ilbers.de>
To: isar-users@googlegroups.com
Subject: [PATCH v2 01/10] base-apt: Add helper class
Date: Thu, 11 Oct 2018 18:52:56 +0200	[thread overview]
Message-ID: <20181011165305.1622-2-mosipov@ilbers.de> (raw)
In-Reply-To: <20181011165305.1622-1-mosipov@ilbers.de>

From: Alexander Smirnov <asmirnov@ilbers.de>

base-apt intended to store original upstream debs to re-use them
later offline. This class helps to populate base-apt with the packages
used during build.

Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
Reviewed-by: Claudius Heine <ch@denx.de>
Signed-off-by: Maxim Yu. Osipov <mosipov@ilbers.de>
---
 meta-isar/conf/layer.conf            |  4 +++
 meta/classes/base-apt-helper.bbclass | 54 ++++++++++++++++++++++++++++++++++++
 2 files changed, 58 insertions(+)
 create mode 100644 meta/classes/base-apt-helper.bbclass

diff --git a/meta-isar/conf/layer.conf b/meta-isar/conf/layer.conf
index cd42f06..22b2ff2 100644
--- a/meta-isar/conf/layer.conf
+++ b/meta-isar/conf/layer.conf
@@ -25,3 +25,7 @@ DEPLOY_DIR_APT ?= "${DEPLOY_DIR}/apt"
 
 # Path to the Isar databases used by `reprepro`
 DEPLOY_DIR_DB ?= "${DEPLOY_DIR}/db"
+
+# Base apt repository paths
+REPO_BASE_DIR ?= "${DEPLOY_DIR}/base-apt/apt"
+REPO_BASE_DB_DIR ?= "${DEPLOY_DIR}/base-apt/db"
diff --git a/meta/classes/base-apt-helper.bbclass b/meta/classes/base-apt-helper.bbclass
new file mode 100644
index 0000000..9c03a7e
--- /dev/null
+++ b/meta/classes/base-apt-helper.bbclass
@@ -0,0 +1,54 @@
+# This software is a part of ISAR.
+# Copyright (C) 2018 ilbers GmbH
+
+DISTRO_NAME ?= "${@ d.getVar('DISTRO', True).split('-')[0]}"
+DISTRO_SUITE ?= "${@ d.getVar('DISTRO', True).split('-')[1]}"
+
+populate_base_apt() {
+    search_dir=$1
+
+    find $search_dir -name '*.deb' | while read package; do
+        # NOTE: due to packages stored by reprepro are not modified, we can
+        # use search by filename to check if package is already in repo. In
+        # addition, m5sums could be compared to ensure, that package is the
+        # same and should not be overwritten. This method is easier and more
+        # robust than querying reprepro by name.
+
+        # Check if this package is taken from Isar-apt, if so - ingore it.
+        base_name=${package##*/}
+        isar_package=$(find ${DEPLOY_DIR_APT}/${DISTRO} -name $base_name)
+        if [ -n "$isar_package" ]; then
+            # Check if MD5 sums are identical. This helps to avoid the case
+            # when packages is overridden from another repo.
+            md1=$(md5sum $package | cut -d ' ' -f 1)
+            md2=$(md5sum $isar_package | cut -d ' ' -f 1)
+            if [ "$md1" = "$md2" ]; then
+                continue
+            fi
+        fi
+
+        # Check if this package is already in base-apt
+        isar_package=$(find ${REPO_BASE_DIR}/${DISTRO_NAME} -name $base_name)
+        if [ -n "$isar_package" ]; then
+            md1=$(md5sum $package | cut -d ' ' -f 1)
+            md2=$(md5sum $isar_package | cut -d ' ' -f 1)
+            if [ "$md1" = "$md2" ]; then
+                continue
+            fi
+
+            # md5sum differs, so remove the package from base-apt
+            name=$($base_name | cut -d '_' -f 1)
+            reprepro -b ${REPO_BASE_DIR}/${DISTRO_NAME} \
+                     --dbdir ${REPO_BASE_DB_DIR}/${DISTRO_NAME} \
+                     -C main -A ${DISTRO_ARCH} \
+                     remove ${DISTRO_SUITE} \
+                     $name
+        fi
+
+        reprepro -b ${REPO_BASE_DIR}/${DISTRO_NAME} \
+                 --dbdir ${REPO_BASE_DB_DIR}/${DISTRO_NAME} \
+                 -C main \
+                 includedeb ${DISTRO_SUITE} \
+                 $package
+    done
+}
-- 
2.11.0


  reply	other threads:[~2018-10-11 16:53 UTC|newest]

Thread overview: 25+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-10-11 16:52 [PATCH v2 00/10] Introduce local apt repo to cache upstream debian packages for offline usage Maxim Yu. Osipov
2018-10-11 16:52 ` Maxim Yu. Osipov [this message]
2018-10-23 16:01   ` [PATCH v2 01/10] base-apt: Add helper class Henning Schild
2018-10-11 16:52 ` [PATCH v2 02/10] meta: Unify path names to local repositories Maxim Yu. Osipov
2018-10-11 16:52 ` [PATCH v2 03/10] base-apt: Introduce base implementation Maxim Yu. Osipov
2018-10-11 16:52 ` [PATCH v2 04/10] isar-boot-strap: Add option to keep cache Maxim Yu. Osipov
2018-10-23 16:06   ` Henning Schild
2018-10-25 14:28   ` Henning Schild
2018-10-11 16:53 ` [PATCH v2 05/10] image: Add cache_base_repo task Maxim Yu. Osipov
2018-10-11 16:53 ` [PATCH v2 06/10] isar-bootstrap: Make possible to reuse the cache Maxim Yu. Osipov
2018-10-23 16:30   ` Henning Schild
2018-10-11 16:53 ` [PATCH v2 07/10] buildchroot: Make it buildable from base-apt Maxim Yu. Osipov
2018-10-11 16:53 ` [PATCH v2 08/10] workaround: Use --allow-unauthenticated working with base-apt Maxim Yu. Osipov
2018-10-23 16:09   ` Henning Schild
2018-10-25 14:33     ` Henning Schild
2018-10-11 16:53 ` [PATCH v2 09/10] local.conf: Add option to use cached base repository Maxim Yu. Osipov
2018-10-11 16:53 ` [PATCH v2 10/10] doc: Creation of local apt repo caching upstream Debian packages Maxim Yu. Osipov
2018-10-17 12:13 ` [PATCH v2 00/10] Introduce local apt repo to cache upstream debian packages for offline usage Maxim Yu. Osipov
2018-10-17 17:11 ` Jan Kiszka
2018-10-17 18:15   ` Maxim Yu. Osipov
2018-10-23  9:56     ` Maxim Yu. Osipov
2018-11-19  8:48       ` Baurzhan Ismagulov
2018-11-19  8:58         ` Jan Kiszka
2018-10-23 16:14 ` Henning Schild
2018-10-29 12:28   ` Baurzhan Ismagulov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20181011165305.1622-2-mosipov@ilbers.de \
    --to=mosipov@ilbers.de \
    --cc=isar-users@googlegroups.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox