Re: [PATCH 3/4] build.sh: Update apt sources

[Alexander Smirnov <asmirnov@ilbers.de>]

On 02/01/2018 04:13 PM, Jan Kiszka wrote:
> On 2018-02-01 12:48, Alexander Smirnov wrote:
>> On 02/01/2018 02:40 PM, Jan Kiszka wrote:
>>> On 2018-02-01 12:29, Alexander Smirnov wrote:
>>>> Trigger apt sources update before building the package to be sure that
>>>> we are in sync with isar-apt content.
>>>>
>>>> Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
>>>> ---
>>>>    meta/recipes-devtools/buildchroot/files/build.sh | 3 +++
>>>>    1 file changed, 3 insertions(+)
>>>>
>>>> diff --git a/meta/recipes-devtools/buildchroot/files/build.sh
>>>> b/meta/recipes-devtools/buildchroot/files/build.sh
>>>> index 77e0fdd..5c59193 100644
>>>> --- a/meta/recipes-devtools/buildchroot/files/build.sh
>>>> +++ b/meta/recipes-devtools/buildchroot/files/build.sh
>>>> @@ -3,6 +3,9 @@
>>>>    # This software is a part of ISAR.
>>>>    # Copyright (C) 2015-2017 ilbers GmbH
>>>>    +# Make sure that we have latest isar-apt content
>>>> +apt-get update
>>>> +
>>>
>>> Can we confine this to the isar-apt repo? Or is there no risk that an
>>> upstream repo update will result in a buildchroot change that is
>>> inconsistent with an earlier build stage?
>>
>> That's a good point, I've already checked it, it seems to be possible like:
>>
>> sudo apt-get update -o Dir::Etc::sourcelist="sources.list.d/${source}" \
>> -o Dir::Etc::sourceparts="-" -o APT::Get::List-Cleanup="0"
>>
>> [https://askubuntu.com/questions/65245/apt-get-update-only-for-a-specific-repository]
>>
>>
>> But I decided that it makes no sense, because target image will be
>> generated later and use upstream apt. So if something has been changed
>> during buildchroot operating, this will affect target image anyway.
> 
> buildchroot shall not deviate from target rootfs. We will eventually
> (rather sooner than later) have to provide it for SDK use, and than you
> definitely don't want surprises, but actually already earlier.
> 

I mean that the issue is not in buildchroot, I'll fix this line to 
update isar-apt only, but this doesn't solve the issue of atomic 
upstream fetch. You can't generate target image using only buildchroot. 
So for example:

  - You've generated buildchroot SDK in Feb.
  - You want to use this SDK and build target image in March.

No warranty that this will work until you prefetch packages for *both* 
target image and buildchroot.

>>
>> Again I think this should be handled outside buildchroot by caching
>> upstream apts.
> 
> Well, do you plan to /always/ cache them? Last time I asked you said we
> need both cases, cached and "live" ;). But even when live, we must keep
> things atomic over a single build at least.

No, I meant a bit different thing: if build_rep assumes to cache 
*everything* exists in Isar tree, I'd like to have an option to disable 
it, I don't want to waste the time by downloading tons of unneeded packages.

In my opinion reproducibility should be always enabled and should cache 
everything needed for *the current* bitbake target, i.e.:

1. $ bitbake example-hello
Isar prefetches everything needed to build example-hello (buildchroot + 
build deps) and use local cache only.

2. $ bitabke isar-image-base
Isar prefetches everything needed to build target image (buildchroot + 
build deps + target image packages) and use local cache only.


Without this Isar reproducibility and build correctness will be based on 
user's hopes and prays :-)

> 
> So, if you can avoid pulling from upstream here, please add that. We can
> still drop it again when the external infrastructure can always
> guarantee atomicity in the repo state.
> 

Ok.

Alex