From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6467463440282681344 X-Received: by 10.46.23.218 with SMTP id 87mr425377ljx.30.1505897361891; Wed, 20 Sep 2017 01:49:21 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 10.28.145.147 with SMTP id t141ls318441wmd.12.canary-gmail; Wed, 20 Sep 2017 01:49:21 -0700 (PDT) X-Google-Smtp-Source: AOwi7QA41zjm6cjocTChdFHGwPBJYnnPSBgUdCLgHuEngsfmNwYD6k7tfV9DPP1tBnGy1JUJ+NnP X-Received: by 10.223.166.39 with SMTP id k36mr291746wrc.19.1505897361542; Wed, 20 Sep 2017 01:49:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1505897361; cv=none; d=google.com; s=arc-20160816; b=Jrmcyr2h4wXZILd0c4Tz0UjI+NvfXDsmjBVu6K0VXwnDtloUOZJQQKng4l+v2fkP3D AGOxq6phAMm0tYYfqhIPpMPkuNNtEKfQeXeiauxNPw5R3006ZeghprP1xi43oFtTsYUL IVXNHnZTslc/OHbKWuikFbPAl/N4nU7X9V+yaCScfXBaQRxlMQNanY9WyisOHUseDUCn BbIGu8itViIlRM/pS3PDfVZ4T2EKS+MDD4GBnVXoizS0OuzdmjYcgoa8HJUEAwFN56bl nZPAifo2c3WLsCTLFNMec0k6W4TDcnQ4+7v3vgVSe85GUxRqVsPpbABi9go0IalaTxgC vJSg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject :arc-authentication-results; bh=Tv0+au3MrqACqmej6gec77mDLUNpVYWXWYCnhypJEGQ=; b=BddLtd6PiQWbiak04pOdpAPS/13UdqkRLSY7p3/BNEM0TdbvWib9muGC6EbmfC0gks pa8xvDxTdSnts8hF9ji59z8lKAMNDXS/7xFCpWna5pDvk7SlGY2GGvA8LMxNglJfpARq ixEniBSf1WxvmteDoDE30iduYHZwEM8OPjcl8Jr9gf+HqWzGeMegFRp3s3c28vzIIdH3 5i3s1YZYbOf1b08w03vAzRNDosr0j7LUHPHbWdoK36kSEpefC7L01Nf0FHEe2/RCX6i1 s7CPOJOqLKTozLd7Xr8tZmiPZFeeKuzNk25bW/yWnsHehubywe7mCfoxEpo8lje5nRhO J8hQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id j4si111787wmi.2.2017.09.20.01.49.21 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 20 Sep 2017 01:49:21 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Received: from [10.0.2.15] ([188.227.110.165]) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id v8K8nIAP013223 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 20 Sep 2017 10:49:20 +0200 Subject: Re: [PATCH v2 3/4] meta-isar-bin: Populate cache To: Henning Schild Cc: isar-users@googlegroups.com References: <20170919122052.28688-1-asmirnov@ilbers.de> <20170919122052.28688-4-asmirnov@ilbers.de> <20170920102253.22199e95@md1em3qc> From: Alexander Smirnov Message-ID: Date: Wed, 20 Sep 2017 11:49:13 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: <20170920102253.22199e95@md1em3qc> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: vMxDElyMOq46 On 09/20/2017 11:22 AM, Henning Schild wrote: > Am Tue, 19 Sep 2017 15:20:51 +0300 > schrieb Alexander Smirnov : > >> Add newly built packages to apt cache. >> >> Signed-off-by: Alexander Smirnov >> --- >> meta/classes/image.bbclass | 9 +++++++++ >> 1 file changed, 9 insertions(+) >> >> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass >> index 9db7371..d59959b 100644 >> --- a/meta/classes/image.bbclass >> +++ b/meta/classes/image.bbclass >> @@ -54,6 +54,15 @@ do_populate[stamp-extra-info] = >> "${DISTRO}-${MACHINE}" # Install Debian packages, that were built >> from sources do_populate() { >> if [ -n "${IMAGE_INSTALL}" ]; then >> + for p in ${IMAGE_INSTALL}; do >> + reprepro --waitforlock 3 \ >> + -b ${DEBCACHEDIR}/${DISTRO} \ >> + --dbdir ${DEBDBDIR}/${DISTRO} \ >> + -C main \ >> + includedeb ${DEBDISTRONAME} \ >> + ${DEPLOY_DIR_DEB}/${p}_*.deb >> + done > > Why not just do that in do_deploy_deb? Maybe DEPLOY_DIR_DEB could go > away because we deploy directly into the repo. This is an attempt to avoid races in reprepro, let me describe the usecase: 1. do_populate() is executed after do_build() 2. If we have several recipes for building, this means that do_populate() could be run in parallel (if you enable parallel jobs for bitbake) 3. --waitforlock does not guarantee that reprepro will take ownership after timeout 4. I agree that DEPLOY_DIR_DEB should go away, but at the moment I think we can use it for a while, until better solution will be found. One approach could be walking through WORKDIRS: for p in ${IMAGE_INSTALL}; do reprepro \ .... ${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${p}-*/*.deb done but I'd like to think if some better solution could be found. > > I found that for kernels built with "make deb-pkg" the "${p}_*.deb" > pattern does not match. At least the way that i named the recipe. > "do_deploy_deb" already works on WORKDIR/*.deb which naturally covers > all packages coming out of a recipe, even if they had names different > from the recipe. Imagine a recipe that downloads multiple .debs from > somewhere and they all have very different names. This works with ${WORKDIR}, where you know that is contains only packages you want (i.e. they are listed in ${IMAGE_INSTALL}). But deploy directory could contain 'unwanted' packages, that for example are left from previous builds. But I think we could ignore this case for now. > > Again the question about the waitforlock, 3 seems only slightly better > than 1. I think we need infinite or a serialization mechanism and 1. Hmm, here you are right. If several architectures in multiconfig will populate ${DEPLOY_DIR_DEB} in parallel, 3 times won't be enough. Will try to implement loop to poll return value from reprepro without waitforlock. Alex > > Henning > >> sudo mkdir -p ${IMAGE_ROOTFS}/deb >> >> for p in ${IMAGE_INSTALL}; do >