From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6558372643829972992 X-Received: by 2002:a50:a8e3:: with SMTP id k90-v6mr250682edc.10.1527075249366; Wed, 23 May 2018 04:34:09 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:aa7:d281:: with SMTP id w1-v6ls10671628edq.5.gmail; Wed, 23 May 2018 04:34:08 -0700 (PDT) X-Google-Smtp-Source: AB8JxZpDeJZg/fcOgOmXooX0Bl72xWka44x/D9kIVTLWkaC4X2uIKVt2Qz8sXDpu+S4whYFhd2tw X-Received: by 2002:aa7:c516:: with SMTP id o22-v6mr1838449edq.6.1527075248903; Wed, 23 May 2018 04:34:08 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1527075248; cv=none; d=google.com; s=arc-20160816; b=B3IOc/ANI5TYJefrR8NGvWoDoMhNcNQe//H6Au3vEjGEMBmM+5k0FNkleQaUCJLOvw Glr3DHEp0YpMJG+Vnzyt2OxePgZ3rzp+GeS4tMRWwl3PpoeKvKXf/Tvtt89uSjKVHSMo xuCHJ19bJK39rcjOYaluRpJCSPVVj9I++z7MTB26Pmdx/IrI6U8qIO1Sjw7ZV/54OJqP N1YO7YDeRv67nQeRNsznWPr41r2tc8Q3C0JQSn6z8qMaNi9H7sGxrT4wVWuKGjeF2edF N0lbda47PVmS4bbko4NgFtU2JcvTluECNkn+Hn4xgBeQHkauxVNqPVJQHF+ZNEUsTxrN KEbA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:references:to:from:subject :arc-authentication-results; bh=3f84eXk/oAt7kk3bmWWSzghskomUfBTesUGAun4jes4=; b=EowbjCX3SiOAIhOjMgUEc7rH4rMudtEIx+PHlPek1j3T6GLGpB2bg39Wvt9RmkzKok +nhiFeD4/SovIwGL7atDk9P9bMtv9xNiCRVSZ8D0MOqpAZGuEHKRdiAbQ227tKsSNLfS 9gVruG/qgd4Di7y3ykSAucCjo10IXzdElXDwsow8loMz95zYLh7yBTE44ak4sugjJAPP tu8vdVTkvYLJfuxsbFCh3CNZqdtHz3OANNvXdOq3/ZKqWpRLLh/pwfwYm+eESAVWYL+7 ukJr0bmPozIgGPKw4mnsSEIzTFy4HircOZ6lhV1WQTr1J4nUQODxEXYKWt0Xepya2hX2 FxlA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Return-Path: Received: from gecko.sbs.de (gecko.sbs.de. [194.138.37.40]) by gmr-mx.google.com with ESMTPS id z2-v6si743360edq.2.2018.05.23.04.34.08 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Wed, 23 May 2018 04:34:08 -0700 (PDT) Received-SPF: pass (google.com: domain of claudius.heine.ext@siemens.com designates 194.138.37.40 as permitted sender) client-ip=194.138.37.40; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 194.138.37.40 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by gecko.sbs.de (8.15.2/8.15.2) with ESMTPS id w4NBY7sI028170 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 23 May 2018 13:34:07 +0200 Received: from [139.25.69.69] (linux-ses-ext02.ppmd.siemens.net [139.25.69.69]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id w4NBY6jx018492 for ; Wed, 23 May 2018 13:34:06 +0200 Subject: Re: Idea for implementing reproducible builds From: Claudius Heine To: isar-users References: <3467a5ec-182e-8c9a-cd19-7ad898323be7@siemens.com> <20180522223224.GE5882@yssyq.radix50.net> <89f104dc-f192-8364-92f2-1345ea11207c@siemens.com> Message-ID: <9d26888a-bd60-31b3-99e2-c024153bbf82@siemens.com> Date: Wed, 23 May 2018 13:34:06 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <89f104dc-f192-8364-92f2-1345ea11207c@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-TUID: wkneaK+joIzI Hi, On 2018-05-23 10:22, [ext] Claudius Heine wrote: >> >> U3.2. debian-mirror exists. Update chosen packages from upstream into >>        debian-mirror. E.g., openssl, optionally its dependencies, >> optionally its >>        dependents. > > Currently that means that the apt index needs to be updated partially. > I don't know if its possible to update this index on a package + > dependency level, but I doubt it. > The result of this is that we need to merge upstream index with our own > and pin all other packages to the old version. > > Even if we just create a complete mirror of all debian mirror, updating > just one package with its dependencies is a serious scripting effort. > > Because of the complexity involved I would postpone this feature. About this point. One way to implement this on top of this implementation could be: 1. Use a script that takes a packagename and repository path and then generates a list of deb packages that describe the current version of this package + its dependencies. 2. This list of packages can be inserted into a bitbake recipe, which downloads those packages, adds them into the isar-apt repository and installs them to the root file system. This way we would be explicit about these partial updates. Of course this script that parses the apt repo might become complex, but there might be some libraries or tools I currently don't know about to help here. TBH I'm mainly an Archlinux user, and with Archlinux partial updates aren't supported, so I am a bit worried about this feature. It may be alright to use this in Debian but I have no idea how good that works in practice. So being very explicit when using it might be a good way to do this. --- I also took a look at aptly [1]. What aptly is missing AFAIK is a way to operate as an apt caching proxy. I mean by this that the mirror command does not download all packages just the index and that the packages will be downloaded by apt requesting them instead of using the 'aptly repo import' command. Maybe if that can be implemented in aptly that might be an alternative to this approach. Partial updates might then be easier to do then. Claudius [1] https://www.aptly.info -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de