From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6519381492065370112 X-Received: by 10.223.190.141 with SMTP id i13mr177623wrh.15.1517911792919; Tue, 06 Feb 2018 02:09:52 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.223.208.133 with SMTP id y5ls1485179wrh.5.gmail; Tue, 06 Feb 2018 02:09:52 -0800 (PST) X-Google-Smtp-Source: AH8x224am6L/1LRGBsd9S17SP2vQKlIPt0wnmNSV6ifbs/GE7WZaCmh2WPrLEqpWCa27odQm7pwS X-Received: by 10.223.176.235 with SMTP id j40mr167400wra.27.1517911792514; Tue, 06 Feb 2018 02:09:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1517911792; cv=none; d=google.com; s=arc-20160816; b=f8491S2+1u4GRZ0v/0u2dINJKIwgk70jjredSlfNS9GxZimPls9ZI7yQFYPWlSw1im sYCRq935wPy6vwrLJsMCNm8LLw4TpAEF+bjZPYZMIUOkmiUdCNnjuYxs9s4Kw0IeOE2w sbIFE+1YAKNDsRt37zEwD0CXk4n1Mf011XF46SaberHbsaMEV5CJrPvCIlc0+t5rLZFT Z3PfKTh0BCic8V7HwTNzU3Z5lT1y5HV3O0940lw/KptdtL5lVKEBK1JtlEu2olH5g/fU rDcvIOyGC8/P5sG3goXe9L6j95SBKpkscdLlyOSHp54e9/ACXCoy7JGLJ8mUc3AkLPo5 Ho7g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature:arc-authentication-results; bh=13ZGqpjTzx21ZgvaTX3QtyeTlvWEolJTBJQ906CCeTE=; b=Ev4xEQ49g45cRTW28QwqQ6eEuuCNrvFtBsvGKiW1IRfIM+nCukbk+9hUXg4FJ4EWzn AhGhNvz37Jw2ebwiIxDguSqyFgaW220DKqCoQfiQZ90h7BTOc2yBmvZh/YbJX8NCRjRQ U4R7wVx+uxuBEpI2IrbEFkaVPIodZg/oIbgzzPZFkF52sCVQqI7sx98nmnw17Gh4abW3 baLjqRfHSgjmaPHmLwqCftuB1MeAm7riOLocmoDBppZZ1cART14F249ItA0/V/Qg24aJ poOR5brMkbXa8M1Z1nnK5EM+/z7smsl7clNC5ABXF3HojsD+1hVaoBFf0Ib70+TcdTUt 6rUA== ARC-Authentication-Results: i=1; gmr-mx.google.com; dkim=pass header.i=@googlemail.com header.s=20161025 header.b=l8LSFeTY; spf=pass (google.com: domain of benbrenson89@googlemail.com designates 2a00:1450:400c:c0c::22e as permitted sender) smtp.mailfrom=benbrenson89@googlemail.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=googlemail.com Return-Path: Received: from mail-wr0-x22e.google.com (mail-wr0-x22e.google.com. [2a00:1450:400c:c0c::22e]) by gmr-mx.google.com with ESMTPS id m23si96483wrb.4.2018.02.06.02.09.52 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 06 Feb 2018 02:09:52 -0800 (PST) Received-SPF: pass (google.com: domain of benbrenson89@googlemail.com designates 2a00:1450:400c:c0c::22e as permitted sender) client-ip=2a00:1450:400c:c0c::22e; Authentication-Results: gmr-mx.google.com; dkim=pass header.i=@googlemail.com header.s=20161025 header.b=l8LSFeTY; spf=pass (google.com: domain of benbrenson89@googlemail.com designates 2a00:1450:400c:c0c::22e as permitted sender) smtp.mailfrom=benbrenson89@googlemail.com; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=googlemail.com Received: by mail-wr0-x22e.google.com with SMTP id w50so1260048wrc.2 for ; Tue, 06 Feb 2018 02:09:52 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlemail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=13ZGqpjTzx21ZgvaTX3QtyeTlvWEolJTBJQ906CCeTE=; b=l8LSFeTY0OrPO5YefbtFyMDv0z5t2WDvearXtzZ1JpUez0fig5Pec7ZiKZ7ADBc1fh YdsvgiOZhse34p5VeygYE+Fh9GVeSI/ux3igGjTdmzEYJH9uYpJ9uZTC5um2CAsjZhGc 4baWcqvdrxTRzqN/xKlmnoj6RraTcxiq8es1ndl/w52M7pf6zaexN+/0CVE6gHLmkdaP eiBeKlE8QgemXg8BX1ki/2wc7UpFirX8Wy2FHpKh9zH7SYNo6W6PnF5SfvVRkRCesRsG 9QNJ4eQ0CLIe/z7tVVdcXimYqWIS0DRXw4JPRn0y4dfojWhP13nbEmQlcX0amDEAtT06 zgUg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=13ZGqpjTzx21ZgvaTX3QtyeTlvWEolJTBJQ906CCeTE=; b=ECcNqvQlB4lrXbJSSpX3hmiikxqDF2pAKqm+UQddW2M4mvdSCl3tuXeyRVWPkLoi1x 60yVSq5zy/DvO7CNVFZOvjlopp9d7Zo1WAcKBc2ar2C+mCSWRgZw4mm7icczdquocMyw HBJTFIH3nHU4+ePkH+OsdVYKIi3lKqN/4dipLw3jT2pWJa0/WAdiFnl5BT63qwhRN6bN o1YkIRq1WLFmVDMBezKaRJTBg+DWJlMtOX7nsGdIAqgorSP48QVU3lq6XzR4omPWh4b2 LUdtdS5xB+meFfH/Lh8WipccKzNem0EZnliFMPHp47DYaiEwjf1y5UmHaM5YxPFnmdrc klrQ== X-Gm-Message-State: APf1xPAsIy0GRhiXvhW8NYoSPBOOifOBLPysTpKhYJlwW1DhmmOoyBRm iOBkDKE8lC6QYscfrlI/dX/K1A== X-Received: by 10.223.187.141 with SMTP id q13mr1771028wrg.65.1517911791877; Tue, 06 Feb 2018 02:09:51 -0800 (PST) Return-Path: Received: from localhost.localdomain (ip-109-41-195-133.web.vodafone.de. [109.41.195.133]) by smtp.gmail.com with ESMTPSA id r5sm13466342wmg.30.2018.02.06.02.09.50 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Tue, 06 Feb 2018 02:09:51 -0800 (PST) From: Benedikt Niedermayr X-Google-Original-From: Benedikt Niedermayr To: isar-users@googlegroups.com Cc: Benedikt Niedermayr Subject: [PATCH 2/6] Added API class for apt cache. Date: Tue, 6 Feb 2018 11:09:35 +0100 Message-Id: <1517911779-30507-3-git-send-email-Benedikt.Niedermayr@mixed-mode.de> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1517911779-30507-1-git-send-email-Benedikt.Niedermayr@mixed-mode.de> References: <1517911779-30507-1-git-send-email-Benedikt.Niedermayr@mixed-mode.de> X-TUID: HYtPMzA3fg+M This class implements a simple shell based API for accessing the underlying functionality of aptly. Signed-off-by: Benedikt Niedermayr --- meta/classes/apt-cache.bbclass | 235 +++++++++++++++++++++++++++++++++++++++++ 1 file changed, 235 insertions(+) create mode 100644 meta/classes/apt-cache.bbclass diff --git a/meta/classes/apt-cache.bbclass b/meta/classes/apt-cache.bbclass new file mode 100644 index 0000000..644e5a2 --- /dev/null +++ b/meta/classes/apt-cache.bbclass @@ -0,0 +1,235 @@ +# This software is a part of ISAR. +# Copyright (C) 2017-2018 Mixed-Mode GmbH +# +# This class implements common functionality for the isar apt cache. + + + +SNAPSHOT_BASENAME="repo-snapshot" +MIRROR_SNAPSHOT_BASENAME="mirror-snapshot" + +ISAR_REPO = "${DEBDISTRONAME}-repo-local" + +ISAR_REPO_PREFIX = "${DEBDISTRONAME}" +ISAR_MIRROR_PREFIX = "${DEBDISTRONAME}-mirror" + +MIRROR = "${DEBDISTRONAME}" +MIRROR_SECURITY = "${DEBDISTRONAME}-security" +MIRROR_UPDATES = "${DEBDISTRONAME}-updates" + +CACHE_CONF_DIR = "${TOPDIR}/apt-cache/${DISTRO}/conf" +CHROOT_CACHE_DIR = "/isar-apt" +CFG_FILE = "${CACHE_CONF_DIR}/aptly.conf" +ISAR_FIRST_BUILD_DONE = "${CACHE_CONF_DIR}/isar_first_build_done" + +# Reproducible build is turned on per default +REPRODUCIBLE_BUILD_ENABLED ?= "1" + + +# Setup a new empty debian mirror. +# $1: The filter for delimiting packages +# $2: Mirror +# $3: Remote URL +# $4: Suite +mirror_create() { + aptly -config=${CFG_FILE} \ + mirror create \ + -architectures="${DISTRO_ARCH}" \ + -filter="$1" \ + -filter-with-deps \ + $2 $3 $4 +} + + +# Update a mirror, which may result in downloading new packages +# $1: Mirror +mirror_update() { + aptly -config=${CFG_FILE} mirror update $1 +} + + +# Create a snapshot from a given mirror. +# $1: Snapshot name. +# $2: Mirror +snapshot_create_from_mirror() { + aptly -config=${CFG_FILE} snapshot create $1 from mirror $2 +} + + +# Create a snapshot from a given repository database +# $1: Snapshot name. +snapshot_create_from_repo() { + aptly -config=${CFG_FILE} \ + snapshot create \ + $1 from repo \ + ${ISAR_REPO} +} + + +# Rename a snapshot +# $1: Old name. +# $2: New name. +snapshot_rename() { + aptly -config=${CFG_FILE} snapshot rename $1 $2 +} + +# Delete a snapshot +# $1: Snapshot name +snapshot_delete() { + aptly -config=${CFG_FILE} \ + snapshot drop \ + $1 || bbnote "No snapshot to delete" +} + + +# Create the actual repository for a given snapshot. +# $1: Snapshot name. +# $2: Publish prefix +# $3: Distribution +snapshot_publish() { + aptly -config=${CFG_FILE} \ + publish snapshot \ + -distribution="$3" \ + $1 \ + $2 +} + + +# Create a new repository database for Isar generated packages. +# $1: Repository database name +repo_db_create() { + aptly -config=${CFG_FILE} \ + repo create \ + -component="main" \ + -distribution="${DEBDISTRONAME}" \ + $1 +} + +# Create a new repository database, but +# use a snapshot as source. +# $1: Repository database name. +# $2: Snapshot name +repo_db_create_from_snapshot() { + aptly -config=${CFG_FILE} \ + repo create \ + -component="main" \ + -distribution="${DEBDISTRONAME}" \ + $1 \ + from snapshot \ + $2 +} + + +# Drop a created repository database silenty. +repo_db_drop() { + aptly -config=${CFG_FILE} \ + repo drop -force \ + ${ISAR_REPO} || bbnote "No db to drop" + +} + + +# Add packages to the local repository database. +# $1: Repository database name. +# $*: Packages to add. +repo_db_add_package() { + repo=$1 + shift + aptly -config=${CFG_FILE} repo add -force-replace $repo $* +} + + +# Delete packages from the repository database. +# $1: Repository database name. +# $*: Packages to delete. +repo_db_delete_package() { + repo=$1 + shift + aptly -config=${CFG_FILE} repo remove $repo $* +} + + +# Publish repository consumed by apt. This function can only be called +# for repositories +# $1: Repository database name +# $2: Path Prefix +repo_db_publish() { + aptly -config=${CFG_FILE} \ + publish repo \ + -architectures="${DISTRO_ARCH}" \ + $1 \ + $2 + +} + + +# Drop an already published repository. +# $1: Path prefix +repo_db_publish_drop() { + aptly -config=${CFG_FILE} \ + publish drop \ + ${DEBDISTRONAME} \ + $1 || bbnote "Nothing to publish" +} + + +# Switch already published repository to new repository. This +# will change the content of the published repository. This function +# can only be called on published repos. +# $1: Prefix +cache_update_repo() { + aptly -config=${CFG_FILE} \ + publish update \ + -force-overwrite \ + ${DEBDISTRONAME} $1 +} + + +# Add packages to the isar cache. +# $1: Repository database name +# $2: Repository prefix +# $*: Packages to add +cache_add_package() { + repo=$1 + prefix=$2 + shift 2 + repo_db_add_package $repo $* + cache_update_repo $prefix +} + + +# Delete a package from the apt cache. +# This function is used to be called within do_clean(all) task. +# $1: Repository database name +# $2: Repository prefix +# $*: Packages to delete +cache_delete_package() { + repo=$1 + prefix=$2 + shift 2 + repo_db_delete_package $repo $* + cache_update_repo $prefix +} + + +# Create a snapshot, from current repository state +cache_create_repo_snapshot() { + repo_db_publish_drop ${ISAR_REPO_PREFIX} + snapshot_delete ${SNAPSHOT_BASENAME}_${MACHINE} + snapshot_create_from_repo ${SNAPSHOT_BASENAME}_${MACHINE} + repo_db_publish ${ISAR_REPO} ${ISAR_REPO_PREFIX} +} + +# Load the last snapshot and publish it +cache_load_snapshot() { + last_snapshot=$(aptly --config=${CFG_FILE} --raw -sort=time snapshot list | grep "$1" | tail -n 1) + + if [ -z "$last_snapshot" ]; then + bbfatal "Requesting snapshot which has never been created. Repository may be in a inconsistent state. Totally new and clean build required." + fi + bbplain "Loading last snapshot: $last_snapshot" + repo_db_publish_drop ${ISAR_REPO_PREFIX} + repo_db_drop + repo_db_create_from_snapshot ${ISAR_REPO} ${SNAPSHOT_BASENAME}_${MACHINE} + repo_db_publish ${ISAR_REPO} ${ISAR_REPO_PREFIX} +} -- 2.7.4