From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6554347564595937280 X-Received: by 2002:a50:f291:: with SMTP id f17-v6mr4662775edm.11.1526296310568; Mon, 14 May 2018 04:11:50 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a50:e84c:: with SMTP id k12-v6ls8280299edn.7.gmail; Mon, 14 May 2018 04:11:50 -0700 (PDT) X-Google-Smtp-Source: AB8JxZqLJkY+cSXBlGr9CM4bs9mlnYLuAL/gdR/YOAqpz+qKZzHs6YzJKDbuhbDg/4tEzilHcymR X-Received: by 2002:a50:c30b:: with SMTP id a11-v6mr4655986edb.8.1526296310174; Mon, 14 May 2018 04:11:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1526296310; cv=none; d=google.com; s=arc-20160816; b=Z5H5fOk9AjDz68tF5Ce2I8YnjKt45lt54kAXiFdZPPz2CAUDo4Kb6EcCf4fO1a6VHz Wzg+SgHmB+ttyjYG3UHFf31gCfEEsWyHNprIQIuAkxSj46/6EXrOHOpyWnuVFwgiTc0K GC6JX8yxEQ89ygElC6TkS4sJwT5KLTe7uBlpJRvppzvDyWJ9nYGI1YzdpNRHl9KxMufy WnCCEUOMDaVMJCSrOG0tePrG738HruD7WbEuq2va9IzsWevV+UqOUC5iY4Z10ThrAm/a NFh/VtMCMb1OiwWqC2AO8ES4+EdrCAujijzHk7HqXbCnATao6sLPe1VZ+Srx4w8KSxQg 4kPA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:to:subject :arc-authentication-results; bh=TQW/RwwSbaZu8i7O2pevoCTPuUxhAibNzMNxLnUyufs=; b=rKgH7y55stJ3LQeFvJSmR3DSQ5FmfhCauq6c91AwKITYjGM/wjtBQBGurtUq/IMLxS nnUUAoUB/Xxp20U1d+itJbh84du7YK+kw1HcFFmxpCXafobhrkapehhssZCufZcfA2Ya 63e8sWyj5BRsormN7q5TT4F8bNwx42mrmjLjO/4gRE/Q+cDz/N2flHuD72f+lZNTIbtd ZFFdXcvp8EhbYK8ZcKJ3F9+DKc338hRCqkzl8HuVKVEZ9SafTFGh01gOW4TSrsQgKZk/ R6eUxFna5L0R8cp6fMlgplZP9VsYxxeZWPPbkiV3xHaOmPLUxvN6T15R/FQjg/uruutg eOKA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id j5-v6si420075edd.5.2018.05.14.04.11.50 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 14 May 2018 04:11:50 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Received: from [10.0.2.15] ([188.227.110.165]) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id w4EBBlrh011405 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Mon, 14 May 2018 13:11:48 +0200 Subject: Re: [PATCH] debootstrap: Add fallback apt cache support To: Claudius Heine , isar-users@googlegroups.com References: <20180511153538.13665-1-asmirnov@ilbers.de> <8de008f9-7bb8-415a-8dcb-08cfa846a701@siemens.com> <049f5e59-ac1c-8a76-754a-e2cf0c82f2cc@ilbers.de> <6128b949-bbc0-9373-b6da-308d8584ae8e@siemens.com> From: Alexander Smirnov Message-ID: <6332767d-c047-939d-2dbd-9e2cb05dd97f@ilbers.de> Date: Mon, 14 May 2018 14:11:41 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: <6128b949-bbc0-9373-b6da-308d8584ae8e@siemens.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 8bit X-TUID: JrbptUbgO6r3 On 05/14/2018 01:33 PM, Claudius Heine wrote: > Hi Alex, > > On 2018-05-14 11:40, Alexander Smirnov wrote: >> >> On 05/14/2018 12:27 PM, Claudius Heine wrote: >>> Hi Alex, >>> >>> so I guess that this patch replaces the 'isar-bootstrap: Increase >>> cache room' one right? >>> >>> On 2018-05-11 17:35, Alexander Smirnov wrote: >>>> Add apt config file to switch caching mechanism from MMap to >>>> array-based. >>>> This needed to provide possibility to build fresh Isar suites on old >>>> hosts. >>>> For example, building Isar stretch on jessie host fails with the >>>> following >>>> error: >>>> >>>>    Hit:1 http://security.debian.org stretch/updates InRelease >>>>    Ign:2 http://ftp.de.debian.org/debian stretch InRelease >>>>    Hit:3 http://ftp.de.debian.org/debian stretch-updates InRelease >>>>    Hit:4 http://ftp.de.debian.org/debian stretch Release >>>>    E: Dynamic MMap ran out of room. Please increase the size of >>>> APT::Cache-Start. Current value: 25165824. (man 5 apt.conf) >>>>    qemu: uncaught target signal 11 (Segmentation fault) - core dumped >>>>    Segmentation fault >>>> >>>> Now Isar performs the following: >>>> 1. Checking if user attempts to build stretch on old kernels >>>> 2. Temporarly installing this apt config file. >>>> 3. Removing this file after image rootfs creation >>>> >>>> So this file will not go to target rootfs. >>>> >>>> Signed-off-by: Alexander Smirnov >>>> --- >>>>   meta-isar/recipes-core/images/isar-image-base.bb        |  2 ++ >>>>   .../isar-bootstrap/files/isar-apt-fallback.conf         |  4 ++++ >>>>   meta/recipes-core/isar-bootstrap/isar-bootstrap.bb      | 17 >>>> ++++++++++++++++- >>>>   3 files changed, 22 insertions(+), 1 deletion(-) >>>>   create mode 100644 >>>> meta/recipes-core/isar-bootstrap/files/isar-apt-fallback.conf >>>> >>>> diff --git a/meta-isar/recipes-core/images/isar-image-base.bb >>>> b/meta-isar/recipes-core/images/isar-image-base.bb >>>> index 00e926a..d65b1f1 100644 >>>> --- a/meta-isar/recipes-core/images/isar-image-base.bb >>>> +++ b/meta-isar/recipes-core/images/isar-image-base.bb >>>> @@ -53,4 +53,6 @@ do_rootfs() { >>>>       sudo rmdir ${IMAGE_ROOTFS}/isar-apt >>>>       sudo umount -l ${IMAGE_ROOTFS}/dev >>>>       sudo umount -l ${IMAGE_ROOTFS}/proc >>>> +    [ -e "${IMAGE_ROOTFS}/etc/apt/apt.conf.d/55isar-fallback.conf" >>>> ] && \ >>>> +        sudo rm >>>> "${IMAGE_ROOTFS}/etc/apt/apt.conf.d/55isar-fallback.conf" || true >>> >>> Why not just 'sudo rm -f >>> "${IMAGE_ROOTFS}/etc/apt/apt.conf.d/55isar-fallback.conf"' ? >>> >> >> Matter of taste, I don't like to remove things that don't exist. > > I guess it is a matter of taste. I do think '|| true' is ugly and If you do think so, why then you use it in your isar-debootstrap implementation? > testing the existence of the file before removing it redundant. > For me it's not too valuable issue, so I could drop this check in v3. Alex > Cheers, > Claudius > >> >>> >>>>   } >>>> diff --git >>>> a/meta/recipes-core/isar-bootstrap/files/isar-apt-fallback.conf >>>> b/meta/recipes-core/isar-bootstrap/files/isar-apt-fallback.conf >>>> new file mode 100644 >>>> index 0000000..12ddbea >>>> --- /dev/null >>>> +++ b/meta/recipes-core/isar-bootstrap/files/isar-apt-fallback.conf >>>> @@ -0,0 +1,4 @@ >>>> +# Switch apt caching mechanism from mmap to array-based. >>>> +# This helps to build fresh Isar suites on old hosts. >>>> + >>>> +APT::Cache-Fallback "1"; >>>> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb >>>> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb >>>> index a38dd88..a3a3a2b 100644 >>>> --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb >>>> +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb >>>> @@ -10,7 +10,9 @@ Description = "Minimal debian root file system" >>>>   LICENSE = "gpl-2.0" >>>>   LIC_FILES_CHKSUM = >>>> "file://${LAYERDIR_isar}/licenses/COPYING.GPLv2;md5=751419260aa954499f7abaabaa882bbe" >>>> >>>>   FILESPATH_prepend := "${THISDIR}/files:" >>>> -SRC_URI = "file://isar-apt.conf" >>>> +SRC_URI = " \ >>>> +    file://isar-apt.conf \ >>>> +    file://isar-apt-fallback.conf" >>>>   PV = "1.0" >>>>   WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}" >>>> @@ -165,6 +167,13 @@ do_bootstrap() { >>>>   } >>>>   addtask bootstrap before do_build after do_apt_config_prepare >>>> +def get_host_release(): >>>> +    import platform >>>> +    rel = platform.release() >>>> +    return rel >>>> + >>>> +HOST_KERN_VER = "${@get_host_release().split('.')[0]}" >>> If you need this variable in just one place. It might be useful to >>> just add it there. >>> >> >> I've added the separate variable, because probably some additional >> version specification could be needed later, like HOST_KERN_MAJOR >> etc., in this case python inlining could be difficult for reading. >> >> But for today I have only one case, so could drop it in v3. >> >> Alex >> >>>> + >>>>   do_apt_config_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}" >>>>   do_apt_config_install() { >>>>       sudo mkdir -p "${ROOTFSDIR}/etc/apt/preferences.d" >>>> @@ -177,6 +186,12 @@ do_apt_config_install() { >>>>       sudo mkdir -p "${ROOTFSDIR}/etc/apt/apt.conf.d" >>>>       sudo install -v -m644 "${WORKDIR}/isar-apt.conf" \ >>>> "${ROOTFSDIR}/etc/apt/apt.conf.d/50isar.conf" >>>> + >>>> +    if [ "${@get_distro_suite(d)}" = "stretch" ] && [ >>>> "${HOST_KERN_VER}" -lt "4" ]; then >>> >>> Like this: >>> >>>      if [ "${@get_distro_suite(d)}" = "stretch" ] && [ >>> "${@get_host_release().split('.')[0]}" -lt "4" ]; then >>> >>> Or is it useful to overwrite this variable somewhere? >>> >>>> +        sudo install -v -m644 "${WORKDIR}/isar-apt-fallback.conf" \ >>>> + "${ROOTFSDIR}/etc/apt/apt.conf.d/55isar-fallback.conf" >>>> +    fi >>>> + >>>>   } >>>>   addtask apt_config_install before do_build after do_bootstrap >>> >>> Apart from those small nitpicks it looks good, >>> Claudius >>> > -- With best regards, Alexander Smirnov ilbers GmbH Baierbrunner Str. 28c D-81379 Munich +49 (89) 122 67 24-0 http://ilbers.de/ Commercial register Munich, HRB 214197 General manager: Baurzhan Ismagulov