From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6732315292542697472 X-Received: by 2002:a7b:cd09:: with SMTP id f9mr38405638wmj.64.1567489302720; Mon, 02 Sep 2019 22:41:42 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:adf:9bd7:: with SMTP id e23ls4572146wrc.7.gmail; Mon, 02 Sep 2019 22:41:42 -0700 (PDT) X-Google-Smtp-Source: APXvYqxM6fkWZdTfNAkaSe7SdVqRJ3PF0GU3jf9GFaU6nKABb2jUUP5JdLt51yTELnnUx3rtOrd9 X-Received: by 2002:a5d:6647:: with SMTP id f7mr10184572wrw.170.1567489302189; Mon, 02 Sep 2019 22:41:42 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1567489302; cv=none; d=google.com; s=arc-20160816; b=LvN6DvLkWZEDbC2sh4Nh158nqlSQk+/0ZNX1gjIH1mJseoUkfuZPBszzSVpw2fkZiO UI6DEIGRNBku3jbUDkTIeP/lzQTYO6zIlpJqMDJJf9Po1vqgHvd+URi54eXBMltBbFBf B8u49i/XJaYS0gqXUGVZ6Twp+EnwICIcRh2uSqS5+aCq3T2sJA8IqeamN10XlMiRtl66 BX5zkdtCnrVYaSwbfKIdcT/wMRM/WU/SQ9gUySXLlsPabBh80UyjMik0lhATzIj/wDg0 VIvyIEB81NRLY5hwO9QvUCwwywtUl42ueqsEIG9Z1jpzEi2eDpXRUQKrHIUOrAIMDWtI L5CA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:mime-version:user-agent :date:message-id:cc:to:subject:from; bh=zi25NJxcIt9XhJSAvUwQYWOqgEuiZXKaQ1cnHdy3EGU=; b=T5A2eUY1CX0SE063fVcqErd9f+yFrUPW1dKIwSj1heO7H6IcQxPEz8sDrVmbA7ZFJL AADEbz6TBQ4Emj55CMCvrnplUWDgV7+bpsANylTqqbmIabZhaNv8AImPIrhVatdSxekZ mIvVwmmBR+BQ4kn7DqPpn+KoJVfyhqLADjG05zIwTxRBAayhWBdOdulshk4aaAFMHdUE /9EFRs0wxfWfI/ZgTddwjAovGcKlRV62Na92XuJgLj83dGciNm1ekG7oMMofBFoRcSlc qCbeKA5FFA/jHB4XYXqs6p0b+5Z4JOo/mTn1AuGrqtFDZwWNAHAguMv9h1oZfBu7J3qG vdog== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from lizzard.sbs.de (lizzard.sbs.de. [194.138.37.39]) by gmr-mx.google.com with ESMTPS id j4si486910wro.5.2019.09.02.22.41.42 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 02 Sep 2019 22:41:42 -0700 (PDT) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.39 as permitted sender) client-ip=194.138.37.39; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 194.138.37.39 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by lizzard.sbs.de (8.15.2/8.15.2) with ESMTPS id x835ffp6018565 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 3 Sep 2019 07:41:41 +0200 Received: from [167.87.40.78] ([167.87.40.78]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id x835fe4P019713; Tue, 3 Sep 2019 07:41:40 +0200 From: Jan Kiszka Subject: [PATCH] rootfs: Take isar-apt shared lock during repository To: isar-users Cc: Claudius Heine , Cedric Hombourger Message-ID: <96728688-e707-9e8f-4f32-9ded7b5300b4@siemens.com> Date: Tue, 3 Sep 2019 07:41:40 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.8.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: k0SMLIPAzoss From: Jan Kiszka We take this lock already for install_imager_deps, but we also need it conceptually for the rootfs installation. Also here, pulling from isar-apt can race with not-yet finished deployments of parallel builds, even if the target package is not shared. See e9a5d8adf6b5 for further details. To avoid holding the lock across the whole lengthy package installation, introduce command flags that define the beginning and the end of the locked section so that do_rootfs_install can manage that. Signed-off-by: Jan Kiszka --- meta/classes/rootfs.bbclass | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass index c7e0435..314efd7 100644 --- a/meta/classes/rootfs.bbclass +++ b/meta/classes/rootfs.bbclass @@ -93,6 +93,7 @@ EOSUDO ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_update" rootfs_install_pkgs_update[weight] = "5" +rootfs_install_pkgs_update[isar-lock] = "acquire-before" rootfs_install_pkgs_update() { sudo -E chroot '${ROOTFSDIR}' /usr/bin/apt-get update \ -o Dir::Etc::sourcelist="sources.list.d/isar-apt.list" \ @@ -108,6 +109,7 @@ rootfs_install_resolvconf() { ROOTFS_INSTALL_COMMAND += "rootfs_install_pkgs_download" rootfs_install_pkgs_download[weight] = "600" +rootfs_install_pkgs_download[isar-lock] = "release-after" rootfs_install_pkgs_download() { sudo -E chroot '${ROOTFSDIR}' \ /usr/bin/apt-get ${ROOTFS_APT_ARGS} --download-only ${ROOTFS_PACKAGES} @@ -153,7 +155,15 @@ python do_rootfs_install() { for cmd in cmds: progress_reporter.next_stage() + + if (d.getVarFlag(cmd, 'isar-lock') or "") == "acquire-before": + lock = bb.utils.lockfile(d.getVar("REPO_ISAR_DIR") + "/isar.lock", + shared=True) + bb.build.exec_func(cmd, d) + + if (d.getVarFlag(cmd, 'isar-lock') or "") == "release-after": + bb.utils.unlockfile(lock) progress_reporter.finish() } addtask rootfs_install before do_rootfs_postprocess after do_unpack -- 2.16.4