From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6554347564595937280 X-Received: by 2002:aa7:c611:: with SMTP id h17-v6mr4618576edq.7.1526294030698; Mon, 14 May 2018 03:33:50 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a50:e84c:: with SMTP id k12-v6ls8236565edn.7.gmail; Mon, 14 May 2018 03:33:50 -0700 (PDT) X-Google-Smtp-Source: AB8JxZp//z6NYYQ4IOxBR9A/lSrH+xipp9gsn1TOj8Wq49LE6hrghhn7iKC71dHbPLkQ21kfOnrL X-Received: by 2002:a50:f291:: with SMTP id f17-v6mr4620024edm.11.1526294030211; Mon, 14 May 2018 03:33:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1526294030; cv=none; d=google.com; s=arc-20160816; b=H4jlrUuw1PqrmtQ3Zz1U1Grg+uB3z7QFGlYqTmPUzuEfNdpwDFObIPKmVPwG7BcbR3 j1S2NisVqzw0YO15O+mFXhPjBjhuRU/pkpxwJNXGQX0WFaBs/BdneXkFRZ0lM6uN2Lm5 sb8Q1W36bFnFH0fB7cbFOyEpEBJ+cTxpADhQ57u5WEs/dCu8O9FUbik20Pk9+NxE8C/n +WnaEcnsfXVnFM0s4qqMPYLyWKgpqL1MKtdaqq3LrUCQIr2u8hQ+1mEkV1bcAiYjd7Gr TYL4iVn7oW+D5RRhlwpP/Nt4H90yzwNVcWNG+6ACVezNBKS0BmAqbGApi0MpgI8Gbo1t 11qA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject :arc-authentication-results; bh=lpDAeSRbjifInYenD0bjBWRTztciLV009U5x2UdU5rc=; b=ivNnzAXKnLuxtNsNiKdaH9R0TlgFe7OP+zggZnsyvYIdnJgEILw47ENACOys4CpWyU ZAByascLWw5/haqdq0Q6KHaetiq44v/srpKlFXp3bKCT7hKhEWyoVRvdTNqY1cdLWoa3 QfLAgHDegNaRuXrvvE1Kz0aLAdEE91uyPZkGTlYsB4jhqE/c33fid2K3T7uMt5YnvihL HLDRQQp02Ds6SPGdOnl1X78AaqGCaVvGwB6JrVFWwjaMVAZEBeAiKiyql/xrNFccqqkL 4YRUxDdOmZPCSjm6k8Ky4fdfXlStzkxtC6mMaYOlbnbo1yaxQCuIUcikun5Ly7GYaC/5 bCrQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Return-Path: Received: from goliath.siemens.de (goliath.siemens.de. [192.35.17.28]) by gmr-mx.google.com with ESMTPS id d24-v6si425113eds.0.2018.05.14.03.33.50 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 14 May 2018 03:33:50 -0700 (PDT) Received-SPF: pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.28 as permitted sender) client-ip=192.35.17.28; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id w4EAXnoX022811 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 May 2018 12:33:49 +0200 Received: from [139.25.69.69] (linux-ses-ext02.ppmd.siemens.net [139.25.69.69]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id w4EAXn1u027418; Mon, 14 May 2018 12:33:49 +0200 Subject: Re: [PATCH] debootstrap: Add fallback apt cache support To: Alexander Smirnov , isar-users@googlegroups.com References: <20180511153538.13665-1-asmirnov@ilbers.de> <8de008f9-7bb8-415a-8dcb-08cfa846a701@siemens.com> <049f5e59-ac1c-8a76-754a-e2cf0c82f2cc@ilbers.de> From: Claudius Heine Message-ID: <6128b949-bbc0-9373-b6da-308d8584ae8e@siemens.com> Date: Mon, 14 May 2018 12:33:49 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <049f5e59-ac1c-8a76-754a-e2cf0c82f2cc@ilbers.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-TUID: 9n6kG1OiaY/r Hi Alex, On 2018-05-14 11:40, Alexander Smirnov wrote: > > On 05/14/2018 12:27 PM, Claudius Heine wrote: >> Hi Alex, >> >> so I guess that this patch replaces the 'isar-bootstrap: Increase >> cache room' one right? >> >> On 2018-05-11 17:35, Alexander Smirnov wrote: >>> Add apt config file to switch caching mechanism from MMap to >>> array-based. >>> This needed to provide possibility to build fresh Isar suites on old >>> hosts. >>> For example, building Isar stretch on jessie host fails with the >>> following >>> error: >>> >>>    Hit:1 http://security.debian.org stretch/updates InRelease >>>    Ign:2 http://ftp.de.debian.org/debian stretch InRelease >>>    Hit:3 http://ftp.de.debian.org/debian stretch-updates InRelease >>>    Hit:4 http://ftp.de.debian.org/debian stretch Release >>>    E: Dynamic MMap ran out of room. Please increase the size of >>> APT::Cache-Start. Current value: 25165824. (man 5 apt.conf) >>>    qemu: uncaught target signal 11 (Segmentation fault) - core dumped >>>    Segmentation fault >>> >>> Now Isar performs the following: >>> 1. Checking if user attempts to build stretch on old kernels >>> 2. Temporarly installing this apt config file. >>> 3. Removing this file after image rootfs creation >>> >>> So this file will not go to target rootfs. >>> >>> Signed-off-by: Alexander Smirnov >>> --- >>>   meta-isar/recipes-core/images/isar-image-base.bb        |  2 ++ >>>   .../isar-bootstrap/files/isar-apt-fallback.conf         |  4 ++++ >>>   meta/recipes-core/isar-bootstrap/isar-bootstrap.bb      | 17 >>> ++++++++++++++++- >>>   3 files changed, 22 insertions(+), 1 deletion(-) >>>   create mode 100644 >>> meta/recipes-core/isar-bootstrap/files/isar-apt-fallback.conf >>> >>> diff --git a/meta-isar/recipes-core/images/isar-image-base.bb >>> b/meta-isar/recipes-core/images/isar-image-base.bb >>> index 00e926a..d65b1f1 100644 >>> --- a/meta-isar/recipes-core/images/isar-image-base.bb >>> +++ b/meta-isar/recipes-core/images/isar-image-base.bb >>> @@ -53,4 +53,6 @@ do_rootfs() { >>>       sudo rmdir ${IMAGE_ROOTFS}/isar-apt >>>       sudo umount -l ${IMAGE_ROOTFS}/dev >>>       sudo umount -l ${IMAGE_ROOTFS}/proc >>> +    [ -e "${IMAGE_ROOTFS}/etc/apt/apt.conf.d/55isar-fallback.conf" ] >>> && \ >>> +        sudo rm >>> "${IMAGE_ROOTFS}/etc/apt/apt.conf.d/55isar-fallback.conf" || true >> >> Why not just 'sudo rm -f >> "${IMAGE_ROOTFS}/etc/apt/apt.conf.d/55isar-fallback.conf"' ? >> > > Matter of taste, I don't like to remove things that don't exist. I guess it is a matter of taste. I do think '|| true' is ugly and testing the existence of the file before removing it redundant. Cheers, Claudius > >> >>>   } >>> diff --git >>> a/meta/recipes-core/isar-bootstrap/files/isar-apt-fallback.conf >>> b/meta/recipes-core/isar-bootstrap/files/isar-apt-fallback.conf >>> new file mode 100644 >>> index 0000000..12ddbea >>> --- /dev/null >>> +++ b/meta/recipes-core/isar-bootstrap/files/isar-apt-fallback.conf >>> @@ -0,0 +1,4 @@ >>> +# Switch apt caching mechanism from mmap to array-based. >>> +# This helps to build fresh Isar suites on old hosts. >>> + >>> +APT::Cache-Fallback "1"; >>> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb >>> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb >>> index a38dd88..a3a3a2b 100644 >>> --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb >>> +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb >>> @@ -10,7 +10,9 @@ Description = "Minimal debian root file system" >>>   LICENSE = "gpl-2.0" >>>   LIC_FILES_CHKSUM = >>> "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe" >>> >>>   FILESPATH_prepend := "${THISDIR}/files:" >>> -SRC_URI = "file://isar-apt.conf" >>> +SRC_URI = " \ >>> +    file://isar-apt.conf \ >>> +    file://isar-apt-fallback.conf" >>>   PV = "1.0" >>>   WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}" >>> @@ -165,6 +167,13 @@ do_bootstrap() { >>>   } >>>   addtask bootstrap before do_build after do_apt_config_prepare >>> +def get_host_release(): >>> +    import platform >>> +    rel = platform.release() >>> +    return rel >>> + >>> +HOST_KERN_VER = "${@get_host_release().split('.')[0]}" >> If you need this variable in just one place. It might be useful to >> just add it there. >> > > I've added the separate variable, because probably some additional > version specification could be needed later, like HOST_KERN_MAJOR etc., > in this case python inlining could be difficult for reading. > > But for today I have only one case, so could drop it in v3. > > Alex > >>> + >>>   do_apt_config_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" >>>   do_apt_config_install() { >>>       sudo mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" >>> @@ -177,6 +186,12 @@ do_apt_config_install() { >>>       sudo mkdir -p "${ROOTFSDIR}/etc/apt/apt.conf.d" >>>       sudo install -v -m644 "${WORKDIR}/isar-apt.conf" \ >>> >>> "${ROOTFSDIR}/etc/apt/apt.conf.d/50isar.conf" >>> + >>> +    if [ "${@get_distro_suite(d)}" = "stretch" ] && [ >>> "${HOST_KERN_VER}" -lt "4" ]; then >> >> Like this: >> >>      if [ "${@get_distro_suite(d)}" = "stretch" ] && [ >> "${@get_host_release().split('.')[0]}" -lt "4" ]; then >> >> Or is it useful to overwrite this variable somewhere? >> >>> +        sudo install -v -m644 "${WORKDIR}/isar-apt-fallback.conf" \ >>> + "${ROOTFSDIR}/etc/apt/apt.conf.d/55isar-fallback.conf" >>> +    fi >>> + >>>   } >>>   addtask apt_config_install before do_build after do_bootstrap >> >> Apart from those small nitpicks it looks good, >> Claudius >> -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de