From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6488313435861811200 X-Received: by 10.25.147.203 with SMTP id w72mr487312lfk.35.1511255222853; Tue, 21 Nov 2017 01:07:02 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.46.29.73 with SMTP id d70ls88199ljd.4.gmail; Tue, 21 Nov 2017 01:07:02 -0800 (PST) X-Google-Smtp-Source: AGs4zMYGWrTCFMAshgZp1k1y/F7WsbnVQu6oHiOXPdmP1IFx22xvPTmdGB5ZF67dHP5n+o4PM23j X-Received: by 10.25.195.140 with SMTP id t134mr499689lff.11.1511255222539; Tue, 21 Nov 2017 01:07:02 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1511255222; cv=none; d=google.com; s=arc-20160816; b=E2bxQyumIoxW38HYFpuEWJ1cL3gTjTq7+dR8syKy0BJVn7Kx4+bEnZW4C41bknzj2g r6P94iCXVR81NLrHevgfoIitoc5tAWChsqtVIyc70yjRjz+3qJCCW6exGowuHg1+5Cyp upg3e1+KwRsEUJI2KwH5JSqBT2dqw2Rvw4D9834tLTsCPXeHKOxsJf2WMCc04FlsJI0N 4tzjZ+kqlwlyAsz2L0nakhnWtlW6V/B5DkAVqRzE7CZx/vQK6fDfnCIFcu7kVOz7dJlQ yBe+cexJuGuzObfzZ3qWNdEUK6pm23q9o5+nkfFws2j3+OOG7f7vWaW80IR18XfyF0Ko a6tA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:arc-authentication-results; bh=TgOH+YAuRjw/AR0NeStUOPflq+xDYD9I/vrbOI5E1ZA=; b=QImHi5upnr/p2fNqaXgEjF51fAiASt582+z6Iit8FY5DfQE1GcKNv0IHPWG97GMDyW x/Q+1fot4hkBTved44Fl6A3NSUaYGx/sL4ccFsAz6SSes+AnDsOqPSuIH4wD9Vm5aKlj hsTyAtjgD1ZfKaAteedwJUYUTWFB62L8fFskvXjQtad3VQPB3auV3p/e5rQ/B3wMZuXe mqFl1npDJlfgl2aQbxDgXKL2x/YrxN7k4mppJjM2RTh54/kMKZCwBNkgEC1oqxhIiXEQ 27kM7/wFNxKXpgfIp4N+FGfM+8PHkbtOYxBaUHpfFkOcCO7GUbzD+fvBHedX86Ft1wct XzCQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id n75si906612ljb.0.2017.11.21.01.07.02 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 21 Nov 2017 01:07:02 -0800 (PST) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Received: from mail1.siemens.de (mail1.siemens.de [139.23.33.14]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id vAL97178027236 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 21 Nov 2017 10:07:01 +0100 Received: from md1em3qc ([139.25.68.40]) by mail1.siemens.de (8.15.2/8.15.2) with ESMTP id vAL97141027543; Tue, 21 Nov 2017 10:07:01 +0100 Date: Tue, 21 Nov 2017 10:07:19 +0100 From: Henning Schild To: Alexander Smirnov Cc: Subject: Re: [PATCH] Map git objects to buildchroot Message-ID: <20171121100719.11d4f2f0@md1em3qc> In-Reply-To: <7e924305-e799-474c-0930-12d169966005@ilbers.de> References: <20171120103728.08714150@md1em3qc> <20171120150641.9645-1-asmirnov@ilbers.de> <20171120163216.494df210@md1em3qc> <20171121092050.1d9adec7@md1em3qc> <7e924305-e799-474c-0930-12d169966005@ilbers.de> X-Mailer: Claws Mail 3.13.2 (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: j0KTVVDRHu90 Am Tue, 21 Nov 2017 11:57:52 +0300 schrieb Alexander Smirnov : > Hi, > > On 11/21/2017 11:20 AM, Henning Schild wrote: > > Am Mon, 20 Nov 2017 18:54:23 +0300 > > schrieb Alexander Smirnov : > > > >> On 11/20/2017 06:32 PM, Henning Schild wrote: > >>> Am Mon, 20 Nov 2017 18:06:41 +0300 > >>> schrieb Alexander Smirnov : > >>> > >>>> Hi Christian, > >>>> > >>>> The patch above has 2 major issues: > >>>> > >>>> 1. It mounts ${GITDIR} to buildchroot during package building. > >>>> This variable is static, so it's the same for each recipe. This > >>>> means that if several packages are building in parallel, the > >>>> folder ${BUILDCHROOT_DIR}/${GITDIR} could be mounted/unmounted > >>>> while there is active I/O operation by another package. To be > >>>> honest, I don't know what could happen, but believe in worst > >>>> case. > >>> > >>> True, that must not happen! > >>> > >>>> 2. Using host configuration in buildchroot could lead to some > >>>> problems. Lets consider the following example: > >>>> - User override ${DL_DIR} to point it to local cache, let's > >>>> say: /home/builder/XXX > >>>> - Isar uses the following path to build package > >>>> YYY: /home/builder/YYY > >>>> - If XXX equals YYY, you will probably experience some > >>>> problems, because your source tree will be overriden by incorect > >>>> mount. > >>>> > >>>> So I'd see the more generic solution like patch below: > >>>> > >>>> 8<-- > >>>> > >>>> dpkg-base: Map git objects to buildchroot > >>>> > >>>> This patch mounts git repository to buildchroot into the > >>>> respective folder: > >>>> > >>>> /git/$REPO_NAME > >>>> > >>>> Also it updates git alternative to correctly point to objects in > >>>> buildchroot. > >>>> > >>>> Signed-off-by: Alexander Smirnov > >>>> --- > >>>> meta/classes/dpkg-base.bbclass | 13 +++++++++++++ > >>>> 1 file changed, 13 insertions(+) > >>>> > >>>> diff --git a/meta/classes/dpkg-base.bbclass > >>>> b/meta/classes/dpkg-base.bbclass index 35af6d5..147cfce 100644 > >>>> --- a/meta/classes/dpkg-base.bbclass > >>>> +++ b/meta/classes/dpkg-base.bbclass > >>>> @@ -19,12 +19,25 @@ dpkg_runbuild() { > >>>> > >>>> # Wrap the function dpkg_runbuild with the bind mount for > >>>> buildroot do_build() { > >>>> + if [ -d ${WORKDIR}/git/.git ]; then > >>>> + OBJ_PATH=$(cat > >>>> ${WORKDIR}/git/.git/objects/info/alternates) > >>>> + REPO_PATH=$(dirname $OBJ_PATH) > >>>> + REPO_NAME=$(basename $REPO_PATH) > >>>> + sudo mkdir -p ${BUILDCHROOT_DIR}/git/$REPO_NAME > >>>> + sudo mount --bind $REPO_PATH > >>>> ${BUILDCHROOT_DIR}/git/$REPO_NAME > >>>> + echo "/git/$REPO_NAME/objects" > > >>>> ${WORKDIR}/git/.git/objects/info/alternates > >>>> + fi > >>>> + > >>>> mkdir -p ${BUILDROOT} > >>>> sudo mount --bind ${WORKDIR} ${BUILDROOT} > >>>> _do_build_cleanup() { > >>>> ret=$? > >>>> sudo umount ${BUILDROOT} 2>/dev/null || true > >>>> sudo rmdir ${BUILDROOT} 2>/dev/null || true > >>>> + if [ -d ${WORKDIR}/git/.git ]; then > >>>> + sudo umount ${BUILDCHROOT_DIR}/git/$REPO_NAME > >>>> + sudo rm -rf ${BUILDCHROOT_DIR}/git/$REPO_NAME > >>>> + fi > >>>> (exit $ret) || bb_exit_handler > >>>> } > >>>> trap '_do_build_cleanup' EXIT > >>> > >>> I would not do this either. There should be one mount in > >>> buildchroot.bb and one umount. Additional umounts could be allowed > >>> in exception handlers, in case they can not call out to functions > >>> from buildchroot. > >> > >> Sorry, didn't get your idea regarding one mount buildchroot.bb, > >> what do you mean? > > > > There should be one mount done from buildchroot.bb before all the > > do_builds of package recipes start. And that should get umounted > > after the last do_build is done. > > Are you speaking about git folder mount only? Or you are speaking > about all the possible mounts related to building process? For this case i am speaking about the git-folder or better the dldir, because that is something that is common to all packages that use buildchroot. Mounts that are packet specific have to stay in the individual recipes. Henning > Alex > > > > >> BTW: we always have option to clone git repository to workspace by > >> running in ${WORKDIR}/git: > >> > >> $ git repack -a > >> > >> in do_build() task. With this, no extra mount/umount logic is > >> needed. > > > > That sounds promising. But i would only do that if git is actually > > the only download-tool that works with such "symlinks". If it is > > not the whole dl-dir should get mounted, not just the git-subdir. > > > >> We could consider this behavior for source 3.0 packages only by > >> using some flag in recipe. > > > > I think that is something we can just do in any case. A flag would > > just complicate things. > > > > Henning > >