From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6488313435861811200 X-Received: by 10.28.26.19 with SMTP id a19mr78238wma.14.1511254681583; Tue, 21 Nov 2017 00:58:01 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.28.152.145 with SMTP id a139ls156376wme.3.canary-gmail; Tue, 21 Nov 2017 00:58:01 -0800 (PST) X-Google-Smtp-Source: AGs4zMag4dpmTQpF18Y6nT3gad2GE5KXg6Z23jSt1pLU3ODabJDkI1A2YIBNMY7GD21aLW1qO0/X X-Received: by 10.28.207.1 with SMTP id f1mr83250wmg.11.1511254681090; Tue, 21 Nov 2017 00:58:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1511254681; cv=none; d=google.com; s=arc-20160816; b=Cct6fYtLqbkAvznXT0XYWSSXhncXLcByrrR/swv143j2gJboa0NkFRCq9zIBdpcrqt 8JBdQizRMhaL6OrkgsA1kM4Uc27pJUVf6aqfyGjMABTALiLIGQH3bKkeaewEw0d6Ae+z rgjvrFzOFoBtbHNEcCiGq51F9ZbKJREKq55OE1SVfPv8o/u7eM/7ruQZLG5DMxEo44Ym ulTvaVIK6JVespLj9yZCDE3FPzjoeL2Ccey5puDpfcTCxI/HZjTURnU/jB3KX41pIvAq mtpiyuVJSIX45VZBaNSxu8QBTPFxJfYvaWSqo3Ptlezs/0+MRwogZw+HSqFdIwd7lYUM CnHw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject :arc-authentication-results; bh=oYk8jbg0psvBswHfFcvOOZ+YgudQ/LRmIUbrLdusJ/g=; b=yoZ89uQZuFbATmFsxUuO03cKWTuJSPszDWi4TV7Aq7CFeHCA9qGwTCU+jqbvQ7pyro eIEsrvEORMV0K2NJ3Mqw7Lj/MFHPqMwALJD1+QLj4kWg1BJhjTajd8kzsPrOhI6IE6oU kR4ik3DUl3405nbARapgmMDVmhTNWHZEZFe+IIV3Krny1mVsY/Sf76NOmas5dXEU2PXi wnTkSgBNtyoUZrN7LucA/WIXv/72ROxcrft3Psxxcwx7j/l1+gx1KZDJ8ARrrmjxQdt5 SuGRID8k+6vfYMMPeEcELGOuQDtlBDdgTc6Ya8CQ5WVYXIco5EZqdzZQNWaiwc0SAQuP wuAw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id b8si919094wrf.5.2017.11.21.00.58.00 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 21 Nov 2017 00:58:00 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Received: from [10.0.2.15] ([188.227.110.165]) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id vAL8vv6e032710 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 21 Nov 2017 09:57:59 +0100 Subject: Re: [PATCH] Map git objects to buildchroot To: Henning Schild Cc: isar-users@googlegroups.com References: <20171120103728.08714150@md1em3qc> <20171120150641.9645-1-asmirnov@ilbers.de> <20171120163216.494df210@md1em3qc> <20171121092050.1d9adec7@md1em3qc> From: Alexander Smirnov Message-ID: <7e924305-e799-474c-0930-12d169966005@ilbers.de> Date: Tue, 21 Nov 2017 11:57:52 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <20171121092050.1d9adec7@md1em3qc> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: f/QwR1TFjU0J Hi, On 11/21/2017 11:20 AM, Henning Schild wrote: > Am Mon, 20 Nov 2017 18:54:23 +0300 > schrieb Alexander Smirnov : > >> On 11/20/2017 06:32 PM, Henning Schild wrote: >>> Am Mon, 20 Nov 2017 18:06:41 +0300 >>> schrieb Alexander Smirnov : >>> >>>> Hi Christian, >>>> >>>> The patch above has 2 major issues: >>>> >>>> 1. It mounts ${GITDIR} to buildchroot during package building. This >>>> variable is static, so it's the same for each recipe. This means >>>> that if several packages are building in parallel, the folder >>>> ${BUILDCHROOT_DIR}/${GITDIR} could be mounted/unmounted while there >>>> is active I/O operation by another package. To be honest, I don't >>>> know what could happen, but believe in worst case. >>> >>> True, that must not happen! >>> >>>> 2. Using host configuration in buildchroot could lead to some >>>> problems. Lets consider the following example: >>>> - User override ${DL_DIR} to point it to local cache, let's >>>> say: /home/builder/XXX >>>> - Isar uses the following path to build package >>>> YYY: /home/builder/YYY >>>> - If XXX equals YYY, you will probably experience some problems, >>>> because your source tree will be overriden by incorect mount. >>>> >>>> So I'd see the more generic solution like patch below: >>>> >>>> 8<-- >>>> >>>> dpkg-base: Map git objects to buildchroot >>>> >>>> This patch mounts git repository to buildchroot into the respective >>>> folder: >>>> >>>> /git/$REPO_NAME >>>> >>>> Also it updates git alternative to correctly point to objects in >>>> buildchroot. >>>> >>>> Signed-off-by: Alexander Smirnov >>>> --- >>>> meta/classes/dpkg-base.bbclass | 13 +++++++++++++ >>>> 1 file changed, 13 insertions(+) >>>> >>>> diff --git a/meta/classes/dpkg-base.bbclass >>>> b/meta/classes/dpkg-base.bbclass index 35af6d5..147cfce 100644 >>>> --- a/meta/classes/dpkg-base.bbclass >>>> +++ b/meta/classes/dpkg-base.bbclass >>>> @@ -19,12 +19,25 @@ dpkg_runbuild() { >>>> >>>> # Wrap the function dpkg_runbuild with the bind mount for >>>> buildroot do_build() { >>>> + if [ -d ${WORKDIR}/git/.git ]; then >>>> + OBJ_PATH=$(cat >>>> ${WORKDIR}/git/.git/objects/info/alternates) >>>> + REPO_PATH=$(dirname $OBJ_PATH) >>>> + REPO_NAME=$(basename $REPO_PATH) >>>> + sudo mkdir -p ${BUILDCHROOT_DIR}/git/$REPO_NAME >>>> + sudo mount --bind $REPO_PATH >>>> ${BUILDCHROOT_DIR}/git/$REPO_NAME >>>> + echo "/git/$REPO_NAME/objects" > >>>> ${WORKDIR}/git/.git/objects/info/alternates >>>> + fi >>>> + >>>> mkdir -p ${BUILDROOT} >>>> sudo mount --bind ${WORKDIR} ${BUILDROOT} >>>> _do_build_cleanup() { >>>> ret=$? >>>> sudo umount ${BUILDROOT} 2>/dev/null || true >>>> sudo rmdir ${BUILDROOT} 2>/dev/null || true >>>> + if [ -d ${WORKDIR}/git/.git ]; then >>>> + sudo umount ${BUILDCHROOT_DIR}/git/$REPO_NAME >>>> + sudo rm -rf ${BUILDCHROOT_DIR}/git/$REPO_NAME >>>> + fi >>>> (exit $ret) || bb_exit_handler >>>> } >>>> trap '_do_build_cleanup' EXIT >>> >>> I would not do this either. There should be one mount in >>> buildchroot.bb and one umount. Additional umounts could be allowed >>> in exception handlers, in case they can not call out to functions >>> from buildchroot. >> >> Sorry, didn't get your idea regarding one mount buildchroot.bb, what >> do you mean? > > There should be one mount done from buildchroot.bb before all the > do_builds of package recipes start. And that should get umounted after > the last do_build is done. Are you speaking about git folder mount only? Or you are speaking about all the possible mounts related to building process? Alex > >> BTW: we always have option to clone git repository to workspace by >> running in ${WORKDIR}/git: >> >> $ git repack -a >> >> in do_build() task. With this, no extra mount/umount logic is needed. > > That sounds promising. But i would only do that if git is actually the > only download-tool that works with such "symlinks". If it is not the > whole dl-dir should get mounted, not just the git-subdir. > >> We could consider this behavior for source 3.0 packages only by using >> some flag in recipe. > > I think that is something we can just do in any case. A flag would just > complicate things. > > Henning >