From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6552866560629800960 X-Received: by 10.28.202.4 with SMTP id a4mr364069wmg.21.1525766226313; Tue, 08 May 2018 00:57:06 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a1c:84c:: with SMTP id 73-v6ls105246wmi.6.canary-gmail; Tue, 08 May 2018 00:57:05 -0700 (PDT) X-Google-Smtp-Source: AB8JxZoqLBjj6SJk2cobYo+1lgOEdeCVFBFd8FWmT7EirM7HXaGM5W5SXrI4npM95lFI8YrpJpK7 X-Received: by 10.28.245.14 with SMTP id t14mr324238wmh.27.1525766225841; Tue, 08 May 2018 00:57:05 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1525766225; cv=none; d=google.com; s=arc-20160816; b=nb9xiSXW4TPH2yD4KXYAUkdTj6WI86gKV96twsREDJdq4dJSWHSYQ47ZGcFE+Q2Dpw 2fe3ElWsP1lWmQWWvMO8Vdfakl18OFXyuE6boOV683U3vRo7i0+tjHW4LkX3L+0GR1lX iGt4+583F9xpDtBzCH6pAD5IOgpKNpV66A6NN7Pd1Kd+YNOGkQfcDYVZKOK7Ds5wttDO tpvjUSX+xxt/Oc0PJwp4QO0tngZDqw/1xZXdB9Q2jq3ztLXt2g9SfhcH41mpLliZB8dM FwaP9N4QS0dv41bz4xelmF9sy+bcHAxmqijVaYxnFOUN5fL4uM3RzOXOVD6ttsbtBsAw JSNQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject :arc-authentication-results; bh=CCKYENhz2sbmcyh3OSJPV0oDmxnYvkAm+9yRyQw/GL8=; b=cEU0FDRIJv6ulmfCJDQbevZOWL8CzxSG1JXR+sHX4XetEunFWOn9c57BCF7zE8PGDV e2TI6qmhNR/wP9E7Z2P5lCt8zAd3z+7ZvcfyVsaTQvJkG8TjXhlS3e4mmn44mwQDjM26 JKogrCso7qDg8pmcGNP6YPsDd1BGszjAZS/js305gozFIMG+fbAmXsI6rWYRTWKxer9j b2jJj/Qjf2fvWSnbhv+YUDp7TC8rvqVt2NrSEJl86M2kvqCV/u9rBHfsB4uJZhvPafXL b21UIMYsfrYvzdJUnuVKkuOfbiXb3qLaEMgfUy79CQRLdKn3BP+ZQHY6s9EDKS/bDH9V aNjA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id s130-v6si459578wms.0.2018.05.08.00.57.05 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 08 May 2018 00:57:05 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Received: from [10.0.2.15] ([188.227.110.165]) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id w487v23E023829 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Tue, 8 May 2018 09:57:04 +0200 Subject: Re: [PATCH] isar-bootstrap: Increase cache room To: Henning Schild Cc: isar-users References: <20180507154836.25738-1-asmirnov@ilbers.de> <52bdb5c0-5f1f-5efc-5de3-9c5a9253ae3b@ilbers.de> <20180507183151.16ff9387@md1pvb1c.ad001.siemens.net> <95cd2859-b074-c8a2-4450-cae977ce9846@ilbers.de> <20180507191213.7b68192b@md1pvb1c.ad001.siemens.net> From: Alexander Smirnov Message-ID: <6fbef55a-276d-43ca-3039-8556c4b1ad8c@ilbers.de> Date: Tue, 8 May 2018 10:56:55 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: <20180507191213.7b68192b@md1pvb1c.ad001.siemens.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: T18vZmKEbTIg On 05/07/2018 08:12 PM, Henning Schild wrote: > Am Mon, 7 May 2018 19:48:16 +0300 > schrieb Alexander Smirnov : > >> Hi, >> >> On 05/07/2018 07:31 PM, Henning Schild wrote: >>> Am Mon, 7 May 2018 18:57:36 +0300 >>> schrieb Alexander Smirnov : >>> >>>> On 05/07/2018 06:48 PM, Alexander Smirnov wrote: >>>>> According to the man page for 'apt.conf', there are three >>>>> variables: >>>>> - Cache-Start: initial size of mmap cache room >>>>> - Cache-Grow: gap for dynamic mmap cache growth >>>>> - Cache-Limit: maximal cache size after growth >>>>> >>>>> If dynamic resize of mmap region is not avaialbe, the apt uses >>>>> pre-allocated Cache-Start room [1] for mmap file. >>>>> >>>>> Building Isar on one of the Debian host with kernel 3.4, I got the >>>>> following problem for 'qemuarm64-stretch' configuration: >>>>> >>>>> 8<-- >>>>> Hit:1 http://security.debian.org stretch/updates InRelease >>>>> Ign:2 http://ftp.de.debian.org/debian stretch InRelease >>>>> Hit:3 http://ftp.de.debian.org/debian stretch-updates InRelease >>>>> Hit:4 http://ftp.de.debian.org/debian stretch Release >>>>> E: Dynamic MMap ran out of room. Please increase the size of >>>>> APT::Cache-Start. Current value: 25165824. (man 5 apt.conf) qemu: >>>>> uncaught target signal 11 (Segmentation fault) - core dumped >>>>> Segmentation fault 8<-- >>>>> >>>>> I have no information, why exactly the room could not be re-sized >>>>> on this system, but it would be better to increase the initial >>>>> room size for apt. This patch increases the default apt cache >>>>> twice. >>>> >>>> 1: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=657085 >>>> >>>> Missed :-( >>> >>> What is your conclusion from that link and what does it mean for the >>> patch? >>> >> >> This just the information I found about dynamic cache increasing. >> Sometimes it could not work. Another fact is that increasing of >> Cache-Limit doesn't help, so seems that mremap really doesn't work. >> >>> I do not like adding magic numbers and changing the default config, >>> especially if the whole reason is to work around a problem only >>> found on outdated build hosts (3.4 really?) >> >> I'm not 100% sure that kernel 3.4 is the source of the problem. Tried >> to investigate, but for now without results. >> >> But if you try to google line: >> >> E: Dynamic MMap ran out of room. Please increase the size of >> APT::Cache-Start >> >> There will be tons of search results, so such problem occurred not >> only on my host. >> >> >> The man page says: >> >> ...on systems with a lot of configured sources it should be >> increased... >> >> so for me it makes sense to increase this room even if there is only >> one problem report. It doesn't break anything. > > Oh yes it breaks the fact that we do not need to care about the value. > If you hardcode one today you might be setting half the default after > the next apt-update. ... because it is a magic number and not > > set(get()*2) Unfortunately this option is not get-able. It's defined as member of class in source code, which is initialized on class creation. There is no getter function to request this variable later. 8<-- apt-pkg/pkgcachegen.cc: map_filesize_t const MapStart = _config->FindI("APT::Cache-Start", 24*1024*1024); ... return new DynamicMMap(*CacheF, Flags, MapStart, MapGrow, MapLimit); 8<-- Anyway, I noticed that build fails later on any 'apt' call in 'buildchroot' and 'image' for this configuration. I was wrong with kernel version, it's 3.16 - the standard Jessie kernel. So for now, I'd like to drop this patch and add supported hosts matrix to Isar. This workaround could be described here. Alex > >> We can't require Isar users to always have up-to-dated host systems. >> If this patch makes all Isar users with old kernels happy, it's >> probably good for Isar in general :-) > > Sure, but there is a cost ... see previous point. > >>> >>> If this patch needs to go in, it is missing the cleanup step. >>> >> >> Sorry, what do you mean here by cleanup step? > > Sorry, misread the patch. I thought it placed a file in the rootfs. > > Henning > >> Alex >> >>> Henning >>> >>>>> Signed-off-by: Alexander Smirnov >>>>> --- >>>>> meta/recipes-core/isar-bootstrap/isar-bootstrap.bb | 4 +++- >>>>> 1 file changed, 3 insertions(+), 1 deletion(-) >>>>> >>>>> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb >>>>> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb index >>>>> a38dd88..4cdefaa 100644 --- >>>>> a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb +++ >>>>> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb @@ -187,7 >>>>> +187,9 @@ do_apt_update() { >>>>> E="${@bb.utils.export_proxies(d)}" >>>>> export DEBIAN_FRONTEND=noninteractive >>>>> - sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y >>>>> + sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y \ >>>>> + -o >>>>> APT::Cache-Start=50331648 + >>>>> sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade >>>>> -y \ -o Debug::pkgProblemResolver=yes >>>>> } >>>>> >>>> >>> >> > -- With best regards, Alexander Smirnov ilbers GmbH Baierbrunner Str. 28c D-81379 Munich +49 (89) 122 67 24-0 http://ilbers.de/ Commercial register Munich, HRB 214197 General manager: Baurzhan Ismagulov