From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6691586504498610176 X-Received: by 2002:a7b:cbcc:: with SMTP id n12mr9392639wmi.167.1558945481930; Mon, 27 May 2019 01:24:41 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a1c:3c0b:: with SMTP id j11ls734033wma.2.canary-gmail; Mon, 27 May 2019 01:24:41 -0700 (PDT) X-Google-Smtp-Source: APXvYqx0mdvLDiTjJeRbX0YGBgqlcTcLRVR1B+jpAS0H6kEwKo4r4Fl9u+WA6hPBkIu4nnp+VXNJ X-Received: by 2002:a1c:6c1a:: with SMTP id h26mr14910473wmc.89.1558945481513; Mon, 27 May 2019 01:24:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1558945481; cv=none; d=google.com; s=arc-20160816; b=imvV6Piela+TyK1iCs+OARQY3oDV+PmztzbNIY8IMlfg46XJJVc7Yl7rW8HhoawrJ8 43BI0zhHs1DWV/VVkB1A0/XdMrHfLwdpnzMN2SCiLpSPfjRXk9gER0Hw5ZyW/07wzvcV jOngqnJUjwEGiExw0kxuchACgMDsKFW7g1ZDmagClZsnkCRO+j17wBGXyITJy7Zt7ggN NuwXldcHx38uD6dvgtTYeFnSG2lBSVMlDdRLFmIi/ZswSTDMhdvDD0F/ZL+mz5MNbnQP IGJD+PTiA4NI7QW4EnhOMsHYDAuHjoYxn8jRR9f1ruwpOwYo7JKYMhd80QYIaMtAQUkp c+Vw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject; bh=xOxjFSdSj8odfTMwAml6Y0q6WtEJGHRYQR5sdofzYnc=; b=I7iTjmSbxFN3MpcYu0ba6UkDwRHdhwWYKzWn9YpfnuPltDCTjuA5xFElR0aHGN7cbF 9IXKP1JOmHOn1I2vTfK6Pv4n4p1iT/urKZRAaxZaIw+yD6CXGJdu5rqiKhDNt/lYRqqu 9pgEw0jaVOKZFeXaKrra95Nrw/2hJYZ+Ckj7bq2aaaZAbrniiDfB9gypS4cCvXmvTQf6 5Y91nRJW5E9YMDcNKK5SYK2/rtmOwdnOpHYPMCeUBtDE5+JjyE+C77p/tClZ6gbPZq67 1iLIOrWr5akR0+rVmNr+X0/t8/2vFmXIt0kMoWosEHv5fCkFwrlSTEMISr8mHqa525Ft 0g8w== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id f83si577511wme.2.2019.05.27.01.24.41 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 May 2019 01:24:41 -0700 (PDT) Received-SPF: pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id x4R8Oe7Q026038 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 27 May 2019 10:24:40 +0200 Received: from [139.25.69.232] (linux-ses-ext02.ppmd.siemens.net [139.25.69.232]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id x4R8OdpD019897; Mon, 27 May 2019 10:24:39 +0200 Subject: Re: [PATCH v4 0/8] pre-processing pipeline and transient package replacement To: "Maxim Yu. Osipov" , isar-users@googlegroups.com Cc: Claudius Heine References: <20190523145521.23050-1-claudius.heine.ext@siemens.com> <7965f1ec-bc24-2c41-ad9a-0d9529b7a640@ilbers.de> <1ba7538a-4656-2e2f-fb85-afd661b75262@ilbers.de> From: Claudius Heine Message-ID: <1a8bdc0e-b71f-0682-e2b4-b3c4147d02e0@siemens.com> Date: Mon, 27 May 2019 10:24:39 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.6.1 MIME-Version: 1.0 In-Reply-To: <1ba7538a-4656-2e2f-fb85-afd661b75262@ilbers.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-TUID: 7rBttYmkkvjH Hi Maxim, On 27/05/2019 10.10, Maxim Yu. Osipov wrote: > Hi Claudius, > > On 5/27/19 9:56 AM, Claudius Heine wrote: >> Hi Maxim, >> >> On 24/05/2019 14.56, Maxim Yu. Osipov wrote: >>> Hi Claudius, >>> >>> CI passed OK. >>> >>> But when I tried to manually test SDK creation procedure (as exactly >>> described in user_manual) it failed: >> >> Have you tested if it does not fail the same way in next when using >> the base-apt repository? > > I've performed the following steps (actually exactly followed current > SDK creation description in user_manual): > > 1) In clean tree with your patches applied, I've enabled cross > compilation in local.conf and launched SDK creation procedure for > qemuarm-stretch > 'build bitbake -c do_populate_sdk > multiconfig:qemuarm-stretch:isar-image-base' > > The build was OK (as expected because such SDK creation build passed in > CI). > > 2) I've mounted needed directories mount_chroot.sh > sdk-debian-stretch-armhf and chrooted into in. > > 3) In SDK chroot commands apt-get update and apt-get install > libhello-dev:armhf failed (logs are below). > > > I've performed the same procedure w/o your patches - it works as > described in user_manual. Well that does not answer my question wherever or not you have enabled ISAR_USE_CACHED_BASE_REPO, in both builds. The logs mention 'file:/base-apt/debian' so I am reasonable sure that you had `ISAR_USE_CACHED_BASE_REPO ="1"` and I just tried that with a filled base-apt in current next resulting in the same error. > > Regards, > Maxim. > > >> regards, >> Claudius >> >>> >>> I've mounted needed directories with >>> 'mount_chroot.sh sdk-debian-stretch-armhf >>> >>> and chroot'ed into sdk: >>> >>> root@tiberius:/#  apt-get install libhello-dev:armhf >>> Reading package lists... Done >>> Building dependency tree >>> Reading state information... Done >>> The following additional packages will be installed: >>>    gcc-6-base:armhf libc6:armhf libgcc1:armhf libhello:armhf >>> Suggested packages: >>>    glibc-doc:armhf locales:armhf >>> The following NEW packages will be installed: >>>    gcc-6-base:armhf libc6:armhf libgcc1:armhf libhello:armhf >>> libhello-dev:armhf >>> 0 upgraded, 5 newly installed, 0 to remove and 0 not upgraded. >>> Need to get 0 B/2,564 kB of archives. >>> After this operation, 8,987 kB of additional disk space will be used. >>> Do you want to continue? [Y/n] >>> Get:1 file:/base-apt/debian stretch/main armhf gcc-6-base armhf >>> 6.3.0-18+deb9u1 [180 kB] >>> Err:1 file:/base-apt/debian stretch/main armhf gcc-6-base armhf >>> 6.3.0-18+deb9u1 >>>    File not found - >>> /base-apt/debian/pool/main/g/gcc-6/gcc-6-base_6.3.0-18+deb9u1_armhf.deb >>> (2: No such file or directory) >>> Get:2 file:/base-apt/debian stretch/main armhf libgcc1 armhf >>> 1:6.3.0-18+deb9u1 [38.5 kB] >>> Err:2 file:/base-apt/debian stretch/main armhf libgcc1 armhf >>> 1:6.3.0-18+deb9u1 >>>    File not found - >>> /base-apt/debian/pool/main/g/gcc-6/libgcc1_6.3.0-18+deb9u1_armhf.deb >>> (2: No such file or directory) >>> Get:3 file:/base-apt/debian stretch/main armhf libc6 armhf >>> 2.24-11+deb9u4 [2,341 kB] >>> Err:3 file:/base-apt/debian stretch/main armhf libc6 armhf >>> 2.24-11+deb9u4 >>>    File not found - >>> /base-apt/debian/pool/main/g/glibc/libc6_2.24-11+deb9u4_armhf.deb (2: >>> No such file or directory) >>> Get:4 file:/isar-apt isar/main armhf libhello armhf 0.1 [2,826 B] >>> Get:5 file:/isar-apt isar/main armhf libhello-dev armhf 0.1 [2,418 B] >>> E: Failed to fetch >>> file:/base-apt/debian/pool/main/g/gcc-6/gcc-6-base_6.3.0-18+deb9u1_armhf.deb >>>   File not found - >>> /base-apt/debian/pool/main/g/gcc-6/gcc-6-base_6.3.0-18+deb9u1_armhf.deb >>> (2: No such file or directory) >>> E: Failed to fetch >>> file:/base-apt/debian/pool/main/g/gcc-6/libgcc1_6.3.0-18+deb9u1_armhf.deb >>>  File not found - >>> /base-apt/debian/pool/main/g/gcc-6/libgcc1_6.3.0-18+deb9u1_armhf.deb >>> (2: No such file or directory) >>> E: Failed to fetch >>> file:/base-apt/debian/pool/main/g/glibc/libc6_2.24-11+deb9u4_armhf.deb File >>> not found - >>> /base-apt/debian/pool/main/g/glibc/libc6_2.24-11+deb9u4_armhf.deb (2: >>> No such file or directory) >>> E: Unable to fetch some archives, maybe run apt-get update or try >>> with --fix-missing? >>> root@tiberius:/# apt-get update >>> Get:1 file:/base-apt/debian stretch InRelease >>> Ign:1 file:/base-apt/debian stretch InRelease >>> Get:2 file:/isar-apt isar InRelease >>> Ign:2 file:/isar-apt isar InRelease >>> Get:3 file:/base-apt/debian stretch Release >>> Ign:3 file:/base-apt/debian stretch Release >>> Get:4 file:/isar-apt isar Release [3,584 B] >>> Get:5 file:/base-apt/debian stretch/main armhf Packages >>> Ign:5 file:/base-apt/debian stretch/main armhf Packages >>> Get:6 file:/base-apt/debian stretch/main all Packages >>> Ign:6 file:/base-apt/debian stretch/main all Packages >>> Get:7 file:/base-apt/debian stretch/main amd64 Packages >>> Ign:7 file:/base-apt/debian stretch/main amd64 Packages >>> Get:8 file:/base-apt/debian stretch/main Translation-en_US >>> Ign:8 file:/base-apt/debian stretch/main Translation-en_US >>> Get:9 file:/base-apt/debian stretch/main Translation-en >>> Ign:9 file:/base-apt/debian stretch/main Translation-en >>> Get:4 file:/isar-apt isar Release [3,584 B] >>> Get:5 file:/base-apt/debian stretch/main armhf Packages >>> Ign:5 file:/base-apt/debian stretch/main armhf Packages >>> Get:6 file:/base-apt/debian stretch/main all Packages >>> Ign:6 file:/base-apt/debian stretch/main all Packages >>> Get:7 file:/base-apt/debian stretch/main amd64 Packages >>> Ign:7 file:/base-apt/debian stretch/main amd64 Packages >>> Get:10 file:/isar-apt isar Release.gpg >>> Ign:10 file:/isar-apt isar Release.gpg >>> Get:8 file:/base-apt/debian stretch/main Translation-en_US >>> Ign:8 file:/base-apt/debian stretch/main Translation-en_US >>> Get:9 file:/base-apt/debian stretch/main Translation-en >>> Ign:9 file:/base-apt/debian stretch/main Translation-en >>> Get:5 file:/base-apt/debian stretch/main armhf Packages >>> Ign:5 file:/base-apt/debian stretch/main armhf Packages >>> Get:6 file:/base-apt/debian stretch/main all Packages >>> Ign:6 file:/base-apt/debian stretch/main all Packages >>> Get:7 file:/base-apt/debian stretch/main amd64 Packages >>> Ign:7 file:/base-apt/debian stretch/main amd64 Packages >>> Get:8 file:/base-apt/debian stretch/main Translation-en_US >>> Ign:8 file:/base-apt/debian stretch/main Translation-en_US >>> Get:9 file:/base-apt/debian stretch/main Translation-en >>> Ign:9 file:/base-apt/debian stretch/main Translation-en >>> Get:5 file:/base-apt/debian stretch/main armhf Packages >>> Ign:5 file:/base-apt/debian stretch/main armhf Packages >>> Get:6 file:/base-apt/debian stretch/main all Packages >>> Ign:6 file:/base-apt/debian stretch/main all Packages >>> Get:7 file:/base-apt/debian stretch/main amd64 Packages >>> Ign:7 file:/base-apt/debian stretch/main amd64 Packages >>> Get:8 file:/base-apt/debian stretch/main Translation-en_US >>> Ign:8 file:/base-apt/debian stretch/main Translation-en_US >>> Get:9 file:/base-apt/debian stretch/main Translation-en >>> Ign:9 file:/base-apt/debian stretch/main Translation-en >>> Get:5 file:/base-apt/debian stretch/main armhf Packages >>> Ign:5 file:/base-apt/debian stretch/main armhf Packages >>> Get:6 file:/base-apt/debian stretch/main all Packages >>> Ign:6 file:/base-apt/debian stretch/main all Packages >>> Get:7 file:/base-apt/debian stretch/main amd64 Packages >>> Ign:7 file:/base-apt/debian stretch/main amd64 Packages >>> Get:8 file:/base-apt/debian stretch/main Translation-en_US >>> Ign:8 file:/base-apt/debian stretch/main Translation-en_US >>> Get:9 file:/base-apt/debian stretch/main Translation-en >>> Ign:9 file:/base-apt/debian stretch/main Translation-en >>> Get:5 file:/base-apt/debian stretch/main armhf Packages >>> Err:5 file:/base-apt/debian stretch/main armhf Packages >>>    File not found - >>> /base-apt/debian/dists/stretch/main/binary-armhf/Packages (2: No such >>> file or directory) >>> Get:6 file:/base-apt/debian stretch/main all Packages >>> Ign:6 file:/base-apt/debian stretch/main all Packages >>> Reading package lists... Done >>> E: Failed to fetch >>> file:/base-apt/debian/dists/stretch/main/binary-armhf/Packages  File >>> not found - /base-apt/debian/dists/stretch/main/binary-armhf/Packages >>> (2: No such file or directory) >>> E: Some index files failed to download. They have been ignored, or >>> old ones used instead. >>> >>> >>> Regards, >>> Maxim. >>> >>> On 5/23/19 4:55 PM, claudius.heine.ext@siemens.com wrote: >>>> From: Claudius Heine >>>> >>>> Hi, >>>> >>>> I decided to heed Hennings advice and order the removal patch before >>>> the split >>>> patch. This way the transient package removal patch becomes more >>>> complete. >>>> >>>> regards, >>>> Claudius >>>> >>>> changes from v3: >>>>   - ordered the transient package removal patch before the split >>>> >>>> changes from v2: >>>>   - added 'copy-package-cache' to the ROOTFS_FEATURES in image.bbclass >>>> >>>> changes from v1: >>>>   - rebased to current next >>>>   - added some more documentation >>>> >>>> Claudius Heine (8): >>>>    meta: remove transient package support >>>>    split up isar-bootstrap helper and implement pre-process pipeline >>>>    meta/classes: add image-locales-extension class >>>>    meta/classes: add image-account-extension class >>>>    doc: update description of image customization >>>>    doc: some fixes >>>>    meta-isar: local.conf.sample: update root password and isar user >>>>      creation >>>>    RECIPE-API-CHANGELOG: update transient package removal + root >>>> password >>>> >>>>   RECIPE-API-CHANGELOG.md                       |  22 ++ >>>>   doc/user_manual.md                            |  53 +++- >>>>   meta-isar/conf/local.conf.sample              |  12 +- >>>>   .../recipes-app/example-raw/files/postinst    |  13 - >>>>   meta/classes/image-account-extension.bbclass  | 257 >>>> ++++++++++++++++++ >>>>   meta/classes/image-locales-extension.bbclass  |  97 +++++++ >>>>   meta/classes/image.bbclass                    |  30 +- >>>>   meta/classes/isar-bootstrap-helper.bbclass    | 143 ---------- >>>>   meta/classes/rootfs.bbclass                   | 176 +++++++++++- >>>>   .../isar-bootstrap/isar-bootstrap.inc         |   5 + >>>>   .../buildchroot/buildchroot-host.bb           |  16 +- >>>>   .../buildchroot/buildchroot-target.bb         |   2 - >>>>   .../buildchroot/buildchroot.inc               |  26 +- >>>>   meta/recipes-devtools/sdkchroot/sdkchroot.bb  |  48 ++-- >>>>   .../isar-cfg-localepurge/files/postinst       |  15 - >>>>   .../isar-cfg-localepurge.bb                   |  70 ----- >>>>   .../isar-cfg-rootpw/files/postinst.tmpl       |  19 -- >>>>   .../isar-cfg-rootpw/isar-cfg-rootpw.bb        |  19 -- >>>>   18 files changed, 663 insertions(+), 360 deletions(-) >>>>   create mode 100644 meta/classes/image-account-extension.bbclass >>>>   create mode 100644 meta/classes/image-locales-extension.bbclass >>>>   delete mode 100644 meta/classes/isar-bootstrap-helper.bbclass >>>>   delete mode 100644 >>>> meta/recipes-support/isar-cfg-localepurge/files/postinst >>>>   delete mode 100644 >>>> meta/recipes-support/isar-cfg-localepurge/isar-cfg-localepurge.bb >>>>   delete mode 100644 >>>> meta/recipes-support/isar-cfg-rootpw/files/postinst.tmpl >>>>   delete mode 100644 >>>> meta/recipes-support/isar-cfg-rootpw/isar-cfg-rootpw.bb >>>> >>> >>> >> > > -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de