From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6467463440282681344 X-Received: by 10.223.172.8 with SMTP id v8mr288389wrc.32.1505897489806; Wed, 20 Sep 2017 01:51:29 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 10.28.134.79 with SMTP id i76ls307450wmd.25.canary-gmail; Wed, 20 Sep 2017 01:51:29 -0700 (PDT) X-Google-Smtp-Source: AOwi7QBDPBq6H5683IyiUq+v10YNWTnX61wFwqoORVRuDGVBIxivJ+ocwiS5Jk15/9n7bSES4q0t X-Received: by 10.28.35.142 with SMTP id j136mr371340wmj.1.1505897489483; Wed, 20 Sep 2017 01:51:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1505897489; cv=none; d=google.com; s=arc-20160816; b=tvQxAlLCHXY7iiyplvub5a8psglWhVbDMqObyotLWt33gNcm5xW6/HeMQb8vIIMmZu IziVybLo0g9Uq72Nq5rcfM1LEKCRyQSYFKXkPmUrpxjUyAx2P7wJV0OChdNY1WChbZZV oXk4Of6W9FIbOGxD/0IUDlVdp2qMDE4fqJ/smtzCjAUTaROkN7//cbERRtR+i+IoR8kM 4Z5zUSya3qWTTbAyvFaOMXayvBDNJ1/oIsssqCoNizDcWjlGqOLK8Ts69jSyFvLwrKKo gOWyA0oTfkrfrfsDOSKev/cVWMx0ZylQwSyld9Va7PQrbRMBOXyBy7JmNNZgn6T5Q7EX UcSA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject :arc-authentication-results; bh=4UQv+rR0FzIub1bbwe0SWiqMDJwU777ErlWWLwxGxgU=; b=MwXkckSacU3381n9u/aqXeekaOuyd4oJ0hbLGwSDQRKatyGOkRwpWYVBZiIHWAeLcP 5qv3gF96gXwowwAwGjF1M/67X32emI/V/Vdt3nPwWJiGRxnhRfFsksR9zjtht4uG1kk8 Vu0Fmy6pcAHVd2IWqbfNWjqdPFpjkrpj9JTxnaYc+62s9rc51tDYIkhmdFwoaRXIqtwF fClJcchU2bNG1xXR11upGwYI9GrYIpr+tT+EoWDjFY0k0QuDD6BGMwjDhMhm/UvHSiK+ 1sU96IJ842kK4TePaPvxc73CV/59xS5PsqoR7HqWzGWMNaKTKS8mgsuPz1mXc72TRjZw 5N2g== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id d82si78884wmd.1.2017.09.20.01.51.29 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 20 Sep 2017 01:51:29 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Received: from [10.0.2.15] ([188.227.110.165]) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id v8K8pQuA013230 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Wed, 20 Sep 2017 10:51:28 +0200 Subject: Re: [PATCH v2 1/4] meta-isar-bin: Add reprepro configs To: Henning Schild Cc: isar-users@googlegroups.com References: <20170919122052.28688-1-asmirnov@ilbers.de> <20170919122052.28688-2-asmirnov@ilbers.de> <20170920095854.33e57f4a@md1em3qc> <22bc2fd0-53b4-52cf-4950-dda1d17755aa@ilbers.de> <20170920103835.2d54d218@md1em3qc> From: Alexander Smirnov Message-ID: <0e293c1c-5b77-b539-e019-3d29adb050ca@ilbers.de> Date: Wed, 20 Sep 2017 11:51:21 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.2.1 MIME-Version: 1.0 In-Reply-To: <20170920103835.2d54d218@md1em3qc> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: pB38rBREm6ap On 09/20/2017 11:38 AM, Henning Schild wrote: > Am Wed, 20 Sep 2017 11:12:11 +0300 > schrieb Alexander Smirnov : > >> On 09/20/2017 10:58 AM, Henning Schild wrote: >>> Am Tue, 19 Sep 2017 15:20:49 +0300 >>> schrieb Alexander Smirnov : >>> >>>> Add template for reprepro configuration file and function that >>>> generates final config for specific deistro. >>>> >>>> Signed-off-by: Alexander Smirnov >>>> --- >>>> meta-isar-bin/conf/layer.conf | 11 +++++++++++ >>>> meta-isar-bin/files/distributions.in | 3 +++ >>>> meta-isar/conf/bblayers.conf.sample | 1 + >>>> meta/classes/image.bbclass | 25 >>>> +++++++++++++++++++++++++ 4 files changed, 40 insertions(+) >>>> create mode 100644 meta-isar-bin/conf/layer.conf >>>> create mode 100644 meta-isar-bin/files/distributions.in >>>> >>>> diff --git a/meta-isar-bin/conf/layer.conf >>>> b/meta-isar-bin/conf/layer.conf new file mode 100644 >>>> index 0000000..3518184 >>>> --- /dev/null >>>> +++ b/meta-isar-bin/conf/layer.conf >>>> @@ -0,0 +1,11 @@ >>>> +# This software is a part of ISAR. >>>> +# Copyright (C) 2017 ilbers GmbH >>>> + >>>> +# Codename of the repository created by the caching class >>>> +DEBDISTRONAME = "isar" >>>> + >>>> +# Path to the caching repository >>>> +DEBCACHEDIR ?= "${LAYERDIR}/apt" >>>> + >>>> +# Path to the configuration files templates used by `reprepro` >>>> +DEBFILESDIR ?= "${LAYERDIR}/files" >>>> diff --git a/meta-isar-bin/files/distributions.in >>>> b/meta-isar-bin/files/distributions.in new file mode 100644 >>>> index 0000000..cd214c6 >>>> --- /dev/null >>>> +++ b/meta-isar-bin/files/distributions.in >>>> @@ -0,0 +1,3 @@ >>>> +Codename: {DISTRO_NAME} >>>> +Architectures: i386 armhf amd64 source >>>> +Components: main >>>> diff --git a/meta-isar/conf/bblayers.conf.sample >>>> b/meta-isar/conf/bblayers.conf.sample index 80867e7..53a362b 100644 >>>> --- a/meta-isar/conf/bblayers.conf.sample >>>> +++ b/meta-isar/conf/bblayers.conf.sample >>>> @@ -8,6 +8,7 @@ BBFILES ?= "" >>>> BBLAYERS ?= " \ >>>> ##ISARROOT##/meta \ >>>> ##ISARROOT##/meta-isar \ >>>> + ##ISARROOT##/meta-isar-bin \ >>>> " >>>> BBLAYERS_NON_REMOVABLE ?= " \ >>>> ##ISARROOT##/meta \ >>>> diff --git a/meta/classes/image.bbclass >>>> b/meta/classes/image.bbclass index 5bf9524..30f241a 100644 >>>> --- a/meta/classes/image.bbclass >>>> +++ b/meta/classes/image.bbclass >>>> @@ -10,6 +10,31 @@ IMAGE_ROOTFS = "${WORKDIR}/rootfs" >>>> >>>> inherit ${IMAGE_TYPE} >>>> >>>> +# Generate apt cache configuration file >>>> +python __anonymous () { >>>> + DISTRO = d.getVar("DISTRO", True) >>>> + DEBFILESDIR = d.getVar("DEBFILESDIR", True) >>>> + DEBCACHEDIR = d.getVar("DEBCACHEDIR", True) >>>> + >>>> + cache_conf_path = os.path.join(DEBCACHEDIR, DISTRO, "conf") >>>> + cache_conf_file = os.path.join(cache_conf_path, >>>> "distributions") >>>> + cache_conf_file_in = os.path.join(DEBFILESDIR, >>>> "distributions.in") + >>>> + import subprocess >>>> + >>>> + if not os.path.exists(cache_conf_path): >>>> + os.makedirs(cache_conf_path) >>>> + >>>> + if not os.path.exists(cache_conf_file): >>>> + f = open(cache_conf_file, "w") >>>> + subprocess.check_call([ >>>> + "sed", >>>> + "-e", "s#{DISTRO_NAME}#" + d.getVar("DEBDISTRONAME", >>>> True) + "#g", >>>> + cache_conf_file_in, >>>> + ], stdout=f) >>>> + f.close() >>>> +} >>> >>> Why is this not just a regular task early in the image chain? >>> >> >> The idea is to avoid races with multiconfig. If I add this as a >> regular task, it could be run in parallel by different multiconfigs, >> so there could occur races in `mkdir` and `sed`. The same is true >> when you try to build several images at the same time: >> >> $ bitbake image1 image2 >> >> IIRC bitbake anonymous tasks are executed during parsing stage, so >> they are serialized. > > I see, but that makes them a very strong measure that you can never > override i.e. in a layer on top. Not sure if that matters but i would > shy away from using the strongest measure if it can be done in a > different way. Agree. > > For the dir how about using newtask[dirs] i guess bitbake will know how > to handle that and it seems to be the bitbake way anyways. Let me try :-) > > And for the sed, if the race happens and two or more seds get run. They > should just produce the same output and not fail if it was already there > > sed "s#{DISTRO_NAME}#" + d.getVar("DEBDISTRONAME", True) + "#g" \ > cache_conf_file_in > cache_conf_file > Ok. > Henning > >> Alex >> >>> Henning >>> >>>> do_populate[stamp-extra-info] = "${DISTRO}-${MACHINE}" >>>> >>>> # Install Debian packages, that were built from sources >>> >