From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6517546778931232768 X-Received: by 10.223.157.147 with SMTP id p19mr243564wre.30.1517495295019; Thu, 01 Feb 2018 06:28:15 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.28.208.199 with SMTP id h190ls1793119wmg.6.canary-gmail; Thu, 01 Feb 2018 06:28:14 -0800 (PST) X-Google-Smtp-Source: AH8x225tmL0qBkmbgI4o22qj94QTQnrFjhieZ2/9gR9Ks+Yu+B1Fr++TBS45hMIlWhHK5lfJ1Akh X-Received: by 10.28.247.5 with SMTP id v5mr3903751wmh.12.1517495294420; Thu, 01 Feb 2018 06:28:14 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1517495294; cv=none; d=google.com; s=arc-20160816; b=EI15olQYZ6iZxmuYuC1PTpvbFwR+Iu38M+9J7b4TVXZuDTmn3hWbxyNHHD7zIFR+Ex HCOfITR7grXLgUM9NwGK6Ba3axJnwL9y+O4QOnlaybDieBSB3hBspJvM2sTCcKIlHDNE DBfYWe6RpyRcA124a3Y+KAfSYA3RlKjFZbyUvLXKVm/RD9tv6k3Uk/kRyyYJePv0cRPU CgoSwinWahvwXL2FaXPgSBCUeXgTW2LgZO9tztC7FeCAFYw3LXPtJewS5Srrknxgm3xS 00q0/Go3uK2iiepxc8xCJgP4QwPPHFTaSvvo3xJ0pUgGFDGFiKG8Wrx3luZv9Rk1dkb8 LYnQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject :arc-authentication-results; bh=XGofKYxWGEvBoK9BYBaYjPRJQ8eqgixoxtvOKvw7hF8=; b=CcMyhVkQYKlI+mxqo5lBIIR1PTroZLRMno6jK8hVtCijKWmkfqMzjc00pvwx4TmcW/ bA4efM2hN3VYGVyh8CthPysyybYxGPj4ikv6L2A4kR+WBSmvpnv8NX5adLHObw66Balw Ry9rA6d00fODRXdxjmIK6+QO3WnKeWRfeNukNHUSEyOyLASgH87b+cel4mmcsVzBf9tH wyHZdyB2j8ALe1QpvUVNAhNJTot2ie2zUmFOZ/ip4fQYDDwG6rWtATebZ7orfOEBokDN J/PAryUjLOdJaxD2eouhymjQwsGhh/hNI7dEF01x3SMMqVf9XuRfBonzdzc3Gn+8N8nH CrFg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id c11si1032949wrb.0.2018.02.01.06.28.14 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Thu, 01 Feb 2018 06:28:14 -0800 (PST) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com Received: from mail1.siemens.de (mail1.siemens.de [139.23.33.14]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id w11ESDrg027856 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Thu, 1 Feb 2018 15:28:14 +0100 Received: from [139.25.68.37] (md1q0hnc.ad001.siemens.net [139.25.68.37] (may be forged)) by mail1.siemens.de (8.15.2/8.15.2) with ESMTP id w11ESDT0014825; Thu, 1 Feb 2018 15:28:13 +0100 Subject: Re: [PATCH 3/4] build.sh: Update apt sources To: Alexander Smirnov , isar-users@googlegroups.com References: <20180201112944.7877-1-asmirnov@ilbers.de> <20180201112944.7877-4-asmirnov@ilbers.de> <6a256d50-9bd9-cad6-e4d6-9fabf19570d8@ilbers.de> <846567dc-92de-8dc6-e532-342dcc00b26c@ilbers.de> From: Jan Kiszka Message-ID: Date: Thu, 1 Feb 2018 15:28:12 +0100 User-Agent: Mozilla/5.0 (X11; U; Linux i686 (x86_64); de; rv:1.8.1.12) Gecko/20080226 SUSE/2.0.0.12-1.1 Thunderbird/2.0.0.12 Mnenhy/0.7.5.666 MIME-Version: 1.0 In-Reply-To: <846567dc-92de-8dc6-e532-342dcc00b26c@ilbers.de> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 8bit X-TUID: kW19ZPABsa56 On 2018-02-01 14:43, Alexander Smirnov wrote: > On 02/01/2018 04:13 PM, Jan Kiszka wrote: >> On 2018-02-01 12:48, Alexander Smirnov wrote: >>> On 02/01/2018 02:40 PM, Jan Kiszka wrote: >>>> On 2018-02-01 12:29, Alexander Smirnov wrote: >>>>> Trigger apt sources update before building the package to be sure that >>>>> we are in sync with isar-apt content. >>>>> >>>>> Signed-off-by: Alexander Smirnov >>>>> --- >>>>>    meta/recipes-devtools/buildchroot/files/build.sh | 3 +++ >>>>>    1 file changed, 3 insertions(+) >>>>> >>>>> diff --git a/meta/recipes-devtools/buildchroot/files/build.sh >>>>> b/meta/recipes-devtools/buildchroot/files/build.sh >>>>> index 77e0fdd..5c59193 100644 >>>>> --- a/meta/recipes-devtools/buildchroot/files/build.sh >>>>> +++ b/meta/recipes-devtools/buildchroot/files/build.sh >>>>> @@ -3,6 +3,9 @@ >>>>>    # This software is a part of ISAR. >>>>>    # Copyright (C) 2015-2017 ilbers GmbH >>>>>    +# Make sure that we have latest isar-apt content >>>>> +apt-get update >>>>> + >>>> >>>> Can we confine this to the isar-apt repo? Or is there no risk that an >>>> upstream repo update will result in a buildchroot change that is >>>> inconsistent with an earlier build stage? >>> >>> That's a good point, I've already checked it, it seems to be possible >>> like: >>> >>> sudo apt-get update -o Dir::Etc::sourcelist="sources.list.d/${source}" \ >>> -o Dir::Etc::sourceparts="-" -o APT::Get::List-Cleanup="0" >>> >>> [https://askubuntu.com/questions/65245/apt-get-update-only-for-a-specific-repository] >>> >>> >>> >>> But I decided that it makes no sense, because target image will be >>> generated later and use upstream apt. So if something has been changed >>> during buildchroot operating, this will affect target image anyway. >> >> buildchroot shall not deviate from target rootfs. We will eventually >> (rather sooner than later) have to provide it for SDK use, and than you >> definitely don't want surprises, but actually already earlier. >> > > I mean that the issue is not in buildchroot, I'll fix this line to > update isar-apt only, but this doesn't solve the issue of atomic > upstream fetch. You can't generate target image using only buildchroot. > So for example: > >  - You've generated buildchroot SDK in Feb. >  - You want to use this SDK and build target image in March. > > No warranty that this will work until you prefetch packages for *both* > target image and buildchroot. That... is true. :) > >>> >>> Again I think this should be handled outside buildchroot by caching >>> upstream apts. >> >> Well, do you plan to /always/ cache them? Last time I asked you said we >> need both cases, cached and "live" ;). But even when live, we must keep >> things atomic over a single build at least. > > No, I meant a bit different thing: if build_rep assumes to cache > *everything* exists in Isar tree, I'd like to have an option to disable > it, I don't want to waste the time by downloading tons of unneeded > packages. > > In my opinion reproducibility should be always enabled and should cache > everything needed for *the current* bitbake target, i.e.: > > 1. $ bitbake example-hello > Isar prefetches everything needed to build example-hello (buildchroot + > build deps) and use local cache only. > > 2. $ bitabke isar-image-base > Isar prefetches everything needed to build target image (buildchroot + > build deps + target image packages) and use local cache only. > > > Without this Isar reproducibility and build correctness will be based on > user's hopes and prays :-) We don't disagree. > >> >> So, if you can avoid pulling from upstream here, please add that. We can >> still drop it again when the external infrastructure can always >> guarantee atomicity in the repo state. >> > > Ok. Let's consider it a "mitigation" measure for "improving" consistency, not able to guarantee it. As you correctly said, as long as we do not cache and freeze our fetches, we are lost with the two rootfs'es we create asynchronously. Thanks, Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux