From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6554347564595937280 X-Received: by 2002:a1c:4188:: with SMTP id o130-v6mr1063117wma.26.1526297102807; Mon, 14 May 2018 04:25:02 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:adf:9d1b:: with SMTP id k27-v6ls4454640wre.1.gmail; Mon, 14 May 2018 04:25:02 -0700 (PDT) X-Google-Smtp-Source: AB8JxZraOu6vE1SmDsBpHoK6NoMgTJV8CDOnBuhASSvaclSMzBvXh3rdrB2B9zvm5WTeboLRs33R X-Received: by 2002:adf:ea87:: with SMTP id s7-v6mr689655wrm.5.1526297102362; Mon, 14 May 2018 04:25:02 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1526297102; cv=none; d=google.com; s=arc-20160816; b=YO42mGal2wbHegxA18/ujNlQWEtjSVuF1aVYk12FfMoKsPbPBt3d7vH+vvedvfQKci 5UrPsO1I/ESLE8iyEzVtnDRR6I6xnjvffFX8eF9vmx5OmctCdAuukLfgNMwwwZrOeG4j DEvZ5eZXc+YkPdJpkR5mhsiAanFSWYpC7pt1l+CxQbx3wosIskyY7xhcg/jBn13hnw5s GZqYJC3Dh+vOjVtHqsfQx5zpjG5WHHrmFax99ZdEFh5hb0sYxkauppgeJ+DKYRQ19xU9 jrm8ZVULizwrBcR16W5QnXZ7UtPq5ksOwXVL3qP9yPFg2riqqY6/j8uN3xKBZgmCDYi+ zM6g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject :arc-authentication-results; bh=M+2u2vs/akLk8NL2oqKcp6D/i30CYZRx0mkG7Yl788A=; b=XF7l0y31xXxmmlhOJ1t6wQMrhCRu1Gbkogzyty5gP11WSZw6Y9GFILy/bE1c0dmlKe W20Fw3R3B8jf9tQ5uXdElo6NzOfBIqYF/AErmpFuwBX6C9/R8quEd4IwSoy8IUd6E8lT Cf77EAnavulVAUDISA9pkh3fRTnmZ0Euy57JVH6SCzzTda+VUxUJQXTWS6ChgRkVTwlN zrYkh/E7Y+d18iZ9mfwAhd2nWLbNTsPJsjF5Rq3vXzGMVyPppCqqsyWtfNvsRvmpnusE n3rSp6z7kkmH/YY7oxrMwGNu0BHdgkc8kAyCnGOyPCfkCnXtcW60457J/haM9L0VeHR5 zsUg== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Return-Path: Received: from david.siemens.de (david.siemens.de. [192.35.17.14]) by gmr-mx.google.com with ESMTPS id e17-v6si391619wri.1.2018.05.14.04.25.02 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 14 May 2018 04:25:02 -0700 (PDT) Received-SPF: pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.14 as permitted sender) client-ip=192.35.17.14; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.14 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by david.siemens.de (8.15.2/8.15.2) with ESMTPS id w4EBP17S028131 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Mon, 14 May 2018 13:25:01 +0200 Received: from [139.25.69.69] (linux-ses-ext02.ppmd.siemens.net [139.25.69.69]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id w4EBP182024462; Mon, 14 May 2018 13:25:01 +0200 Subject: Re: [PATCH] debootstrap: Add fallback apt cache support To: Alexander Smirnov , isar-users@googlegroups.com References: <20180511153538.13665-1-asmirnov@ilbers.de> <8de008f9-7bb8-415a-8dcb-08cfa846a701@siemens.com> <049f5e59-ac1c-8a76-754a-e2cf0c82f2cc@ilbers.de> <6128b949-bbc0-9373-b6da-308d8584ae8e@siemens.com> <6332767d-c047-939d-2dbd-9e2cb05dd97f@ilbers.de> From: Claudius Heine Message-ID: <29d3f697-edec-40e6-8c57-1b53dcb9e3ba@siemens.com> Date: Mon, 14 May 2018 13:25:01 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <6332767d-c047-939d-2dbd-9e2cb05dd97f@ilbers.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-TUID: 2sm26YDkUN2J On 2018-05-14 13:11, Alexander Smirnov wrote: > > > On 05/14/2018 01:33 PM, Claudius Heine wrote: >> Hi Alex, >> >> On 2018-05-14 11:40, Alexander Smirnov wrote: >>> >>> On 05/14/2018 12:27 PM, Claudius Heine wrote: >>>> Hi Alex, >>>> >>>> so I guess that this patch replaces the 'isar-bootstrap: Increase >>>> cache room' one right? >>>> >>>> On 2018-05-11 17:35, Alexander Smirnov wrote: >>>>> Add apt config file to switch caching mechanism from MMap to >>>>> array-based. >>>>> This needed to provide possibility to build fresh Isar suites on >>>>> old hosts. >>>>> For example, building Isar stretch on jessie host fails with the >>>>> following >>>>> error: >>>>> >>>>>    Hit:1 http://security.debian.org stretch/updates InRelease >>>>>    Ign:2 http://ftp.de.debian.org/debian stretch InRelease >>>>>    Hit:3 http://ftp.de.debian.org/debian stretch-updates InRelease >>>>>    Hit:4 http://ftp.de.debian.org/debian stretch Release >>>>>    E: Dynamic MMap ran out of room. Please increase the size of >>>>> APT::Cache-Start. Current value: 25165824. (man 5 apt.conf) >>>>>    qemu: uncaught target signal 11 (Segmentation fault) - core dumped >>>>>    Segmentation fault >>>>> >>>>> Now Isar performs the following: >>>>> 1. Checking if user attempts to build stretch on old kernels >>>>> 2. Temporarly installing this apt config file. >>>>> 3. Removing this file after image rootfs creation >>>>> >>>>> So this file will not go to target rootfs. >>>>> >>>>> Signed-off-by: Alexander Smirnov >>>>> --- >>>>>   meta-isar/recipes-core/images/isar-image-base.bb        |  2 ++ >>>>>   .../isar-bootstrap/files/isar-apt-fallback.conf         |  4 ++++ >>>>>   meta/recipes-core/isar-bootstrap/isar-bootstrap.bb      | 17 >>>>> ++++++++++++++++- >>>>>   3 files changed, 22 insertions(+), 1 deletion(-) >>>>>   create mode 100644 >>>>> meta/recipes-core/isar-bootstrap/files/isar-apt-fallback.conf >>>>> >>>>> diff --git a/meta-isar/recipes-core/images/isar-image-base.bb >>>>> b/meta-isar/recipes-core/images/isar-image-base.bb >>>>> index 00e926a..d65b1f1 100644 >>>>> --- a/meta-isar/recipes-core/images/isar-image-base.bb >>>>> +++ b/meta-isar/recipes-core/images/isar-image-base.bb >>>>> @@ -53,4 +53,6 @@ do_rootfs() { >>>>>       sudo rmdir ${IMAGE_ROOTFS}/isar-apt >>>>>       sudo umount -l ${IMAGE_ROOTFS}/dev >>>>>       sudo umount -l ${IMAGE_ROOTFS}/proc >>>>> +    [ -e "${IMAGE_ROOTFS}/etc/apt/apt.conf.d/55isar-fallback.conf" >>>>> ] && \ >>>>> +        sudo rm >>>>> "${IMAGE_ROOTFS}/etc/apt/apt.conf.d/55isar-fallback.conf" || true >>>> >>>> Why not just 'sudo rm -f >>>> "${IMAGE_ROOTFS}/etc/apt/apt.conf.d/55isar-fallback.conf"' ? >>>> >>> >>> Matter of taste, I don't like to remove things that don't exist. >> >> I guess it is a matter of taste. I do think '|| true' is ugly and > > If you do think so, why then you use it in your isar-debootstrap > implementation? You mean like this? sudo umount -l "${ROOTFSDIR}/dev" || true I still find that ugly, but in may cases unavoidable. Bash/Shell is just a ugly language IMO. Just because something is ugly doesn't mean it should or can be avoided all the time. > >> testing the existence of the file before removing it redundant. >> > > For me it's not too valuable issue, so I could drop this check in v3. Ok, thanks. Claudius > > Alex > >> Cheers, >> Claudius >> >>> >>>> >>>>>   } >>>>> diff --git >>>>> a/meta/recipes-core/isar-bootstrap/files/isar-apt-fallback.conf >>>>> b/meta/recipes-core/isar-bootstrap/files/isar-apt-fallback.conf >>>>> new file mode 100644 >>>>> index 0000000..12ddbea >>>>> --- /dev/null >>>>> +++ b/meta/recipes-core/isar-bootstrap/files/isar-apt-fallback.conf >>>>> @@ -0,0 +1,4 @@ >>>>> +# Switch apt caching mechanism from mmap to array-based. >>>>> +# This helps to build fresh Isar suites on old hosts. >>>>> + >>>>> +APT::Cache-Fallback "1"; >>>>> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb >>>>> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb >>>>> index a38dd88..a3a3a2b 100644 >>>>> --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb >>>>> +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb >>>>> @@ -10,7 +10,9 @@ Description = "Minimal debian root file system" >>>>>   LICENSE = "gpl-2.0" >>>>>   LIC_FILES_CHKSUM = >>>>> "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe" >>>>> >>>>>   FILESPATH_prepend := "${THISDIR}/files:" >>>>> -SRC_URI = "file://isar-apt.conf" >>>>> +SRC_URI = " \ >>>>> +    file://isar-apt.conf \ >>>>> +    file://isar-apt-fallback.conf" >>>>>   PV = "1.0" >>>>>   WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}" >>>>> @@ -165,6 +167,13 @@ do_bootstrap() { >>>>>   } >>>>>   addtask bootstrap before do_build after do_apt_config_prepare >>>>> +def get_host_release(): >>>>> +    import platform >>>>> +    rel = platform.release() >>>>> +    return rel >>>>> + >>>>> +HOST_KERN_VER = "${@get_host_release().split('.')[0]}" >>>> If you need this variable in just one place. It might be useful to >>>> just add it there. >>>> >>> >>> I've added the separate variable, because probably some additional >>> version specification could be needed later, like HOST_KERN_MAJOR >>> etc., in this case python inlining could be difficult for reading. >>> >>> But for today I have only one case, so could drop it in v3. >>> >>> Alex >>> >>>>> + >>>>>   do_apt_config_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" >>>>>   do_apt_config_install() { >>>>>       sudo mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" >>>>> @@ -177,6 +186,12 @@ do_apt_config_install() { >>>>>       sudo mkdir -p "${ROOTFSDIR}/etc/apt/apt.conf.d" >>>>>       sudo install -v -m644 "${WORKDIR}/isar-apt.conf" \ >>>>> "${ROOTFSDIR}/etc/apt/apt.conf.d/50isar.conf" >>>>> + >>>>> +    if [ "${@get_distro_suite(d)}" = "stretch" ] && [ >>>>> "${HOST_KERN_VER}" -lt "4" ]; then >>>> >>>> Like this: >>>> >>>>      if [ "${@get_distro_suite(d)}" = "stretch" ] && [ >>>> "${@get_host_release().split('.')[0]}" -lt "4" ]; then >>>> >>>> Or is it useful to overwrite this variable somewhere? >>>> >>>>> +        sudo install -v -m644 "${WORKDIR}/isar-apt-fallback.conf" \ >>>>> + "${ROOTFSDIR}/etc/apt/apt.conf.d/55isar-fallback.conf" >>>>> +    fi >>>>> + >>>>>   } >>>>>   addtask apt_config_install before do_build after do_bootstrap >>>> >>>> Apart from those small nitpicks it looks good, >>>> Claudius >>>> >> > -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de