From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6517546778931232768 X-Received: by 10.80.243.144 with SMTP id g16mr13214836edm.11.1517484611626; Thu, 01 Feb 2018 03:30:11 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.80.169.98 with SMTP id m31ls4304857edc.0.gmail; Thu, 01 Feb 2018 03:30:10 -0800 (PST) X-Google-Smtp-Source: AH8x227Elglo6Dom5JCBVP+GSYmrAOe15i0AoyJFkHLj7om36Qajzi1vdnVP37zSp4KXV8kpNTkP X-Received: by 10.80.144.177 with SMTP id c46mr13220295eda.8.1517484610941; Thu, 01 Feb 2018 03:30:10 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1517484610; cv=none; d=google.com; s=arc-20160816; b=odE+qgxhk/XGsiBvMgBJGKuljP79ogfV491kMmJo97l7rgfo3Ozndnq0gKEEUdTTVP vLoyU58mVUONdddb8761es0tN+vzh4X/ynGwZJDnOQrQRj6xEisr0npWBHsznhlqtjlq uYINhgnw3rBESKykA7nZKL0lreSYseD8se508LJErmsvHqlgb9lQdTsGdoA/kNYB0fyI jvmOjC7uyQoFNSKh0xf2+Dz28xddHoWirkkT517X4hAa/1QU4X3S8xPuSltkHaW9pNAg +tHhJEtG4UsBZqgwmN1+HenOd3nstIaWLvRSCnlLRrIxphQdOm4Cm+yglLiiXJxFKR0c sbKQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=I23c+ubExpHSUP184mVulFzg87Ef5atedP6wI2tNOTo=; b=h5cIAzqjufd7ojD5AkUgxL1HkLOVyIz5mQBinUbBztHwJDHjVC0PfULD90GZz10L3V 2qMvIFd0/9ED8x2GBrPgBYKokm3QhkmcJc9mf1vttvBGKmdxoKRFH4xlTjxfc2h+HpC8 Cm40pGveIGFQ9bi9L+yUhwm/C2km9b5WHDSMZSWXXUPjvOnBHGuMBzZOp+B4iXZ/N0qT KQu5oUi/bHPKqCtp8lbOg03riEmYyEK9dDvucRwfhTocaOBmSZ8zafYTRcGoEbRv62fb RYdvgWdFSWu/RxsifaGLUsCPMHl8D7D0r/xJFQzQDu/weUy0Oi7w6GLq/faqqGIyb6tg Ipow== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id z17si1805107edz.1.2018.02.01.03.30.10 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 01 Feb 2018 03:30:10 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Received: from localhost.localdomain ([188.227.110.165]) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id w11BTpZW011589 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Thu, 1 Feb 2018 12:30:01 +0100 From: Alexander Smirnov To: isar-users@googlegroups.com Cc: Alexander Smirnov Subject: [PATCH 1/4] isar-apt: Introduce separate recipe Date: Thu, 1 Feb 2018 14:29:41 +0300 Message-Id: <20180201112944.7877-2-asmirnov@ilbers.de> X-Mailer: git-send-email 2.9.5 In-Reply-To: <20180201112944.7877-1-asmirnov@ilbers.de> References: <20180201112944.7877-1-asmirnov@ilbers.de> X-TUID: LFLMuPQH3eDS Motivation: in current implementation whole isar-apt logic are done as a part of image class. This actually not the optimal way due to the various issues like: - isar-apt should be singletone in current tree despite on amount of images to be built. - isar-apt API could be possible extended, for example by adding do_export(). - *-image is not the mandatory target to build. User may build just example-hello, but isar-apt should handle this. So there should be separate recipe which is responsibe for isar-apt management. Also isar-apt should be a static part of core Isar like buildchroot. This patch drops isar-apt from image class, and introduces isar-apt recipe. Signed-off-by: Alexander Smirnov --- meta-isar/conf/local.conf.sample | 4 -- .../recipes-core/images/files/distributions.in | 3 - meta/classes/dpkg-base.bbclass | 12 +++- meta/classes/image.bbclass | 67 +--------------------- .../isar-apt/files/distributions.in | 3 + meta/recipes-devtools/isar-apt/isar-apt.bb | 29 ++++++++++ 6 files changed, 43 insertions(+), 75 deletions(-) delete mode 100644 meta-isar/recipes-core/images/files/distributions.in create mode 100644 meta/recipes-devtools/isar-apt/files/distributions.in create mode 100644 meta/recipes-devtools/isar-apt/isar-apt.bb diff --git a/meta-isar/conf/local.conf.sample b/meta-isar/conf/local.conf.sample index 33d988e..2ae43e7 100644 --- a/meta-isar/conf/local.conf.sample +++ b/meta-isar/conf/local.conf.sample @@ -158,7 +158,3 @@ IMAGE_INSTALL = "example-hello example-raw" # # Default parallel jobs for bitbake: BB_NUMBER_THREADS = "4" - -# -# Number of attempts to try to get reprepro lock for access to apt cache -REPREPRO_LOCK_ATTEMPTS = "16" diff --git a/meta-isar/recipes-core/images/files/distributions.in b/meta-isar/recipes-core/images/files/distributions.in deleted file mode 100644 index cd214c6..0000000 --- a/meta-isar/recipes-core/images/files/distributions.in +++ /dev/null @@ -1,3 +0,0 @@ -Codename: {DISTRO_NAME} -Architectures: i386 armhf amd64 source -Components: main diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass index 4941f9b..026028f 100644 --- a/meta/classes/dpkg-base.bbclass +++ b/meta/classes/dpkg-base.bbclass @@ -31,11 +31,17 @@ do_build() { _do_build_cleanup } -# Install package to dedicated deploy directory +# Install package to Isar-apt do_deploy_deb() { - install -m 644 ${WORKDIR}/*.deb ${DEPLOY_DIR_DEB}/ + reprepro -b ${DEPLOY_DIR_APT}/${DISTRO} \ + --dbdir ${DEPLOY_DIR_DB}/${DISTRO} \ + -C main \ + includedeb ${DEBDISTRONAME} \ + ${WORKDIR}/*.deb } addtask deploy_deb after do_build do_deploy_deb[dirs] = "${DEPLOY_DIR_DEB}" -do_deploy_deb[stamp-extra-info] = "${MACHINE}" +do_deploy_deb[stamp-extra-info] = "${DISTRO}-${MACHINE}" +do_deploy_deb[lockfiles] = "${DEPLOY_DIR_APT}/isar.lock" +do_deploy_deb[depends] = "isar-apt:do_cache_config" diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass index e2cb01b..cbd74b3 100644 --- a/meta/classes/image.bbclass +++ b/meta/classes/image.bbclass @@ -17,70 +17,6 @@ KERNEL_IMAGE ?= "${@get_image_name(d, 'vmlinuz')}" INITRD_IMAGE ?= "${@get_image_name(d, 'initrd.img')}" inherit ${IMAGE_TYPE} -call_reprepro() { - for i in $(seq 1 ${REPREPRO_LOCK_ATTEMPTS}); do - # According to `sh` manual page, shell exit statuses fall between - # 0-255. The EEXIST error code is (-17), so casting to usigned 8-bit - # integer results value (239). - eexist=$(python -c 'import errno; print(256-errno.EEXIST)') - retval="0" - reprepro $* || retval="$?" - - # If reprepro has failed to get database lock, it returns EEXIST code. - # In this case we continue trying to get lock until max amount of - # attempts is reached. - if [ $retval -eq $eexist ]; then - bbwarn "Failed to get reprepro lock, trying again..." - sleep 5 - else - break - fi - done - - if [ $retval -ne 0 ]; then - bbfatal "reprepro failed" - fi -} - -CACHE_CONF_DIR = "${DEPLOY_DIR_APT}/${DISTRO}/conf" -do_cache_config[dirs] = "${CACHE_CONF_DIR}" -do_cache_config[stamp-extra-info] = "${DISTRO}" - -# Generate reprepro config for current distro if it doesn't exist. Once it's -# generated, this task should do nothing. -do_cache_config() { - if [ ! -e "${CACHE_CONF_DIR}/distributions" ]; then - sed -e "s#{DISTRO_NAME}#"${DEBDISTRONAME}"#g" \ - ${FILESDIR}/distributions.in > ${CACHE_CONF_DIR}/distributions - fi - - path_cache="${DEPLOY_DIR_APT}/${DISTRO}" - path_databases="${DEPLOY_DIR_DB}/${DISTRO}" - - if [ ! -d "${path_databases}" ]; then - call_reprepro -b ${path_cache} \ - --dbdir ${path_databases} \ - export ${DEBDISTRONAME} - fi -} - -addtask cache_config before do_populate - -do_populate[stamp-extra-info] = "${DISTRO}-${MACHINE}" - -# Populate Isar apt repository by newly built packages -do_populate() { - if [ -n "${IMAGE_INSTALL}" ]; then - call_reprepro -b ${DEPLOY_DIR_APT}/${DISTRO} \ - --dbdir ${DEPLOY_DIR_DB}/${DISTRO} \ - -C main \ - includedeb ${DEBDISTRONAME} \ - ${DEPLOY_DIR_DEB}/*.deb - fi -} - -addtask populate before do_build after do_unpack -do_populate[deptask] = "do_deploy_deb" do_rootfs[stamp-extra-info] = "${MACHINE}-${DISTRO}" @@ -88,7 +24,8 @@ do_rootfs() { die "No root filesystem function defined, please implement in your recipe" } -addtask rootfs before do_build after do_populate +addtask rootfs before do_build after do_unpack +do_rootfs[deptask] = "do_deploy_deb" do_copy_boot_files() { KERNEL_IMAGE=${@get_image_name(d, 'vmlinuz')} diff --git a/meta/recipes-devtools/isar-apt/files/distributions.in b/meta/recipes-devtools/isar-apt/files/distributions.in new file mode 100644 index 0000000..cd214c6 --- /dev/null +++ b/meta/recipes-devtools/isar-apt/files/distributions.in @@ -0,0 +1,3 @@ +Codename: {DISTRO_NAME} +Architectures: i386 armhf amd64 source +Components: main diff --git a/meta/recipes-devtools/isar-apt/isar-apt.bb b/meta/recipes-devtools/isar-apt/isar-apt.bb new file mode 100644 index 0000000..9c31d12 --- /dev/null +++ b/meta/recipes-devtools/isar-apt/isar-apt.bb @@ -0,0 +1,29 @@ +# This software is a part of ISAR. +# Copyright (C) 2015-2017 ilbers GmbH + +SRC_URI = "file://distributions.in" + +CACHE_CONF_DIR = "${DEPLOY_DIR_APT}/${DISTRO}/conf" +do_cache_config[dirs] = "${CACHE_CONF_DIR}" +do_cache_config[stamp-extra-info] = "${DISTRO}" +do_cache_config[lockfiles] = "${DEPLOY_DIR_APT}/isar.lock" + +# Generate reprepro config for current distro if it doesn't exist. Once it's +# generated, this task should do nothing. +do_cache_config() { + if [ ! -e "${CACHE_CONF_DIR}/distributions" ]; then + sed -e "s#{DISTRO_NAME}#"${DEBDISTRONAME}"#g" \ + ${WORKDIR}/distributions.in > ${CACHE_CONF_DIR}/distributions + fi + + path_cache="${DEPLOY_DIR_APT}/${DISTRO}" + path_databases="${DEPLOY_DIR_DB}/${DISTRO}" + + if [ ! -d "${path_databases}" ]; then + reprepro -b ${path_cache} \ + --dbdir ${path_databases} \ + export ${DEBDISTRONAME} + fi +} + +addtask cache_config after do_unpack before do_build -- 2.1.4