From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6656673633724792832 Date: Mon, 11 Feb 2019 05:33:18 -0800 (PST) From: chombourger@gmail.com To: isar-users Message-Id: In-Reply-To: <0ff54328-5b30-e645-619e-0eaac9c184b5@siemens.com> References: <20190211093324.1444-1-henning.schild@siemens.com> <20190211103526.3d92babf@md1za8fc.ad001.siemens.net> <0ff54328-5b30-e645-619e-0eaac9c184b5@siemens.com> Subject: Re: [PATCH] dpkg-base: apt-get "update" before "source" MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="----=_Part_1255_943877528.1549891999101" X-Google-Token: EJ7zheMF5kKChuRv4L00 X-Google-IP: 192.94.31.2 X-TUID: esJcLiDyUj1Z ------=_Part_1255_943877528.1549891999101 Content-Type: multipart/alternative; boundary="----=_Part_1256_641904780.1549891999101" ------=_Part_1256_641904780.1549891999101 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Hi Jan, Will need to recheck things here but there should be no .list files referencing public repos when building from the local apt cache (and therefore when you are wanting a reproducible build) My local patch essentially does a full apt-get update but I now need to check if I did not alter the expected behavior :) Cedric On Monday, February 11, 2019 at 1:53:09 PM UTC+1, Jan Kiszka wrote: > > On 11.02.19 10:35, [ext] Henning Schild wrote: > > Here i see an often repeating pattern. That "apt-get update" is now > > present in many entry points to the buildchroot. > > I guess we should factor it out and put it into a central place. And > > the rule of thumb probably is ... whenever you use anything apt, > > apt-get update before you do ... > > If you are only talking about updating our locally maintained repo (like > below), > that is fine to factor out and reuse. However, we must not update against > public > repos after the initial pulling, in order to ensure we have a consistent > package > set along the whole build. > > Jan > > > > > Henning > > > > Am Mon, 11 Feb 2019 10:33:24 +0100 > > schrieb Henning Schild >: > > > >> From: Henning Schild > > >> > >> When rebuilding we can run into an inconsistent view where buildchroot > >> assumes it could download the sources of a modified upstream package. > >> After a "reprepro ... remove" we always need to "apt-get update" to > >> not operate on an old version of the metadata. > >> > >> Signed-off-by: Henning Schild > > >> --- > >> meta/classes/dpkg-base.bbclass | 4 ++++ > >> 1 file changed, 4 insertions(+) > >> > >> diff --git a/meta/classes/dpkg-base.bbclass > >> b/meta/classes/dpkg-base.bbclass index 175dc80..5425df8 100644 > >> --- a/meta/classes/dpkg-base.bbclass > >> +++ b/meta/classes/dpkg-base.bbclass > >> @@ -31,6 +31,10 @@ do_apt_fetch() { > >> fi > >> dpkg_do_mounts > >> E="${@ bb.utils.export_proxies(d)}" > >> + sudo -E chroot ${BUILDCHROOT_DIR} /usr/bin/apt-get update \ > >> + -o > >> Dir::Etc::sourcelist="sources.list.d/isar-apt.list" \ > >> + -o Dir::Etc::sourceparts="-" \ > >> + -o APT::Get::List-Cleanup="0" > >> sudo -E chroot --userspec=$( id -u ):$( id -g ) > >> ${BUILDCHROOT_DIR} \ sh -c 'cd ${PP} && apt-get -y source ${SRC_APT}' > >> dpkg_undo_mounts > > > > -- > Siemens AG, Corporate Technology, CT RDA IOT SES-DE > Corporate Competence Center Embedded Linux > ------=_Part_1256_641904780.1549891999101 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: quoted-printable
Hi Jan,

Will need to recheck things her= e but there should be no .list files referencing public repos when building= from the local apt cache (and therefore when you are wanting a reproducibl= e build)
My local patch essentially does a full apt-get update bu= t I now need to check if I did not alter the expected behavior :)

Cedric

On Monday, February 11, 2019 at 1:53:09 PM UT= C+1, Jan Kiszka wrote:
On 11.02= .19 10:35, [ext] Henning Schild wrote:
> Here i see an often repeating pattern. That "apt-get update&q= uot; is now
> present in many entry points to the buildchroot.
> I guess we should factor it out and put it into a central place. A= nd
> the rule of thumb probably is ... whenever you use anything apt,
> apt-get update before you do ...

If you are only talking about updating our locally maintained repo (lik= e below),=20
that is fine to factor out and reuse. However, we must not update again= st public=20
repos after the initial pulling, in order to ensure we have a consisten= t package=20
set along the whole build.

Jan

>=20
> Henning
>=20
> Am Mon, 11 Feb 2019 10:33:24 +0100
> schrieb Henning Schild <henning...@siemens.com>:
>=20
>> From: Henning Schild <henning...@siemens.com>
>>
>> When rebuilding we can run into an inconsistent view where bui= ldchroot
>> assumes it could download the sources of a modified upstream p= ackage.
>> After a "reprepro ... remove" we always need to &quo= t;apt-get update" to
>> not operate on an old version of the metadata.
>>
>> Signed-off-by: Henning Schild <henning...@siemens.com>
>> ---
>> =C2=A0 meta/classes/dpkg-base.bbclass | 4 ++++
>> =C2=A0 1 file changed, 4 insertions(+)
>>
>> diff --git a/meta/classes/dpkg-base.bbclass
>> b/meta/classes/dpkg-base.bbclass index 175dc80..5425df8 1= 00644
>> --- a/meta/classes/dpkg-base.bbclass
>> +++ b/meta/classes/dpkg-base.bbclass
>> @@ -31,6 +31,10 @@ do_apt_fetch() {
>> =C2=A0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0fi
>> =C2=A0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0dpkg_do= _mounts
>> =C2=A0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0E=3D&qu= ot;${@ bb.utils.export_proxies(d)}"
>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0sudo -E chroo= t ${BUILDCHROOT_DIR} /usr/bin/apt-get update \
>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0-o
>> Dir::Etc::sourcelist=3D"sources.list.d/isar-apt.list= " \
>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0-o Dir::Etc::sourceparts=3D"-"= ; \
>> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0-o APT::Get::List-Cleanup=3D"0&quo= t;
>> =C2=A0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0sudo -E= chroot --userspec=3D$( id -u ):$( id -g )
>> ${BUILDCHROOT_DIR} \ sh -c 'cd ${PP} && apt-get -y= source ${SRC_APT}'
>> =C2=A0 =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0dpkg_un= do_mounts
>=20

--=20
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
------=_Part_1256_641904780.1549891999101-- ------=_Part_1255_943877528.1549891999101--