From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6552866560629800960 X-Received: by 2002:a19:9f8f:: with SMTP id i137-v6mr2285667lfe.3.1525769787515; Tue, 08 May 2018 01:56:27 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:7e0b:: with SMTP id z11-v6ls75451ljc.0.gmail; Tue, 08 May 2018 01:56:27 -0700 (PDT) X-Google-Smtp-Source: AB8JxZqh14Kwfq+pS+qIPTni5W4lQ5AmnHkPseFdLRukJfutJMJuczHz9LmuqVH3PdUixajeIy5o X-Received: by 2002:a2e:9595:: with SMTP id w21-v6mr1502061ljh.5.1525769787032; Tue, 08 May 2018 01:56:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1525769786; cv=none; d=google.com; s=arc-20160816; b=B5Kl8w6Z189x2KcjCc2xrhqK/u3JNEKTgbXFBVJVBhGxxWzZIf+KOECN+H7/7oO4c+ SIGahXXHaEIHsLgncANQk0wbYoShgvB/KfU5heYaiib5zDorweGh0bm1+jI4u7cIZeIa Z9HspWevBxfyIC/uw6jIn738P1OX173gzakHmvY+npxyo9YhXKyQr2OyLbEzaEfcg82i 9+A7eZRPrdTnObqe5QDhHfm5W432kFUQcT1whV/lLpaae/fDUXmao6js5HPvMjIk8ksA Bs2sfxXu0LJI6lfjWukOvK3oWSOzqG+ksX8WTVnfmDOQWpkiNILIBjWeNfAJbn0dd/9P 8pOw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date:arc-authentication-results; bh=krlzklUsj6mgYM7Yz9SP5885uUd4uJnyEzzTH7+TIPQ=; b=SQR6O8PTA1xg/ViK3mCrra+esJmr5vnmOUJKcicDuGkviG0p0qi2GFz6cv7HTLFMAV rEGeSSMepHn3UhC9RXywwN/fM52vdxCNTLnJgnW6Pl+rfLR1Kb9ZIOLVySSFc+7NH4L9 7b+7uN0kGwP9RGf7gr1Fg1WCQqLSjssXw2nfyzNbAuV5IyW8YGN0eYYtXshhnyAD6MZ7 vsdOW0yUodsOVb7kNtJsZIZ2WAxqtOljDfHvVC6kIy1PxMB8VS6emsA8RcKx1BXvxaTO JxX8We98Vbn+LsTh8Mfc7usm5FGMl+89/+B0b+cYBOAQCnXj3IwLeSL52XaAidnEnH0l ssNQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id i130-v6si666134lfi.5.2018.05.08.01.56.26 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 08 May 2018 01:56:26 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=henning.schild@siemens.com Received: from mail1.siemens.de (mail1.siemens.de [139.23.33.14]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id w488uQw4015074 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 8 May 2018 10:56:26 +0200 Received: from md1pvb1c.ad001.siemens.net (md1pvb1c.ad001.siemens.net [139.25.68.40]) by mail1.siemens.de (8.15.2/8.15.2) with ESMTP id w488uQTi011588; Tue, 8 May 2018 10:56:26 +0200 Date: Tue, 8 May 2018 10:56:25 +0200 From: Henning Schild To: Alexander Smirnov Cc: isar-users Subject: Re: [PATCH] isar-bootstrap: Increase cache room Message-ID: <20180508105625.1bc1ff88@md1pvb1c.ad001.siemens.net> In-Reply-To: <6fbef55a-276d-43ca-3039-8556c4b1ad8c@ilbers.de> References: <20180507154836.25738-1-asmirnov@ilbers.de> <52bdb5c0-5f1f-5efc-5de3-9c5a9253ae3b@ilbers.de> <20180507183151.16ff9387@md1pvb1c.ad001.siemens.net> <95cd2859-b074-c8a2-4450-cae977ce9846@ilbers.de> <20180507191213.7b68192b@md1pvb1c.ad001.siemens.net> <6fbef55a-276d-43ca-3039-8556c4b1ad8c@ilbers.de> X-Mailer: Claws Mail 3.15.0-dirty (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: 3ZNWKsTwNl3T Am Tue, 8 May 2018 10:56:55 +0300 schrieb Alexander Smirnov : > On 05/07/2018 08:12 PM, Henning Schild wrote: > > Am Mon, 7 May 2018 19:48:16 +0300 > > schrieb Alexander Smirnov : > > > >> Hi, > >> > >> On 05/07/2018 07:31 PM, Henning Schild wrote: > >>> Am Mon, 7 May 2018 18:57:36 +0300 > >>> schrieb Alexander Smirnov : > >>> > >>>> On 05/07/2018 06:48 PM, Alexander Smirnov wrote: > >>>>> According to the man page for 'apt.conf', there are three > >>>>> variables: > >>>>> - Cache-Start: initial size of mmap cache room > >>>>> - Cache-Grow: gap for dynamic mmap cache growth > >>>>> - Cache-Limit: maximal cache size after growth > >>>>> > >>>>> If dynamic resize of mmap region is not avaialbe, the apt uses > >>>>> pre-allocated Cache-Start room [1] for mmap file. > >>>>> > >>>>> Building Isar on one of the Debian host with kernel 3.4, I got > >>>>> the following problem for 'qemuarm64-stretch' configuration: > >>>>> > >>>>> 8<-- > >>>>> Hit:1 http://security.debian.org stretch/updates InRelease > >>>>> Ign:2 http://ftp.de.debian.org/debian stretch InRelease > >>>>> Hit:3 http://ftp.de.debian.org/debian stretch-updates InRelease > >>>>> Hit:4 http://ftp.de.debian.org/debian stretch Release > >>>>> E: Dynamic MMap ran out of room. Please increase the size of > >>>>> APT::Cache-Start. Current value: 25165824. (man 5 apt.conf) > >>>>> qemu: uncaught target signal 11 (Segmentation fault) - core > >>>>> dumped Segmentation fault 8<-- > >>>>> > >>>>> I have no information, why exactly the room could not be > >>>>> re-sized on this system, but it would be better to increase the > >>>>> initial room size for apt. This patch increases the default apt > >>>>> cache twice. > >>>> > >>>> 1: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=657085 > >>>> > >>>> Missed :-( > >>> > >>> What is your conclusion from that link and what does it mean for > >>> the patch? > >>> > >> > >> This just the information I found about dynamic cache increasing. > >> Sometimes it could not work. Another fact is that increasing of > >> Cache-Limit doesn't help, so seems that mremap really doesn't work. > >> > >>> I do not like adding magic numbers and changing the default > >>> config, especially if the whole reason is to work around a > >>> problem only found on outdated build hosts (3.4 really?) > >> > >> I'm not 100% sure that kernel 3.4 is the source of the problem. > >> Tried to investigate, but for now without results. > >> > >> But if you try to google line: > >> > >> E: Dynamic MMap ran out of room. Please increase the size of > >> APT::Cache-Start > >> > >> There will be tons of search results, so such problem occurred not > >> only on my host. > >> > >> > >> The man page says: > >> > >> ...on systems with a lot of configured sources it should be > >> increased... > >> > >> so for me it makes sense to increase this room even if there is > >> only one problem report. It doesn't break anything. > > > > Oh yes it breaks the fact that we do not need to care about the > > value. If you hardcode one today you might be setting half the > > default after the next apt-update. ... because it is a magic number > > and not > > > > set(get()*2) > > Unfortunately this option is not get-able. It's defined as member of > class in source code, which is initialized on class creation. There > is no getter function to request this variable later. > > 8<-- > apt-pkg/pkgcachegen.cc: > > map_filesize_t const MapStart = _config->FindI("APT::Cache-Start", > 24*1024*1024); > ... > return new DynamicMMap(*CacheF, Flags, MapStart, MapGrow, MapLimit); > 8<-- > > Anyway, I noticed that build fails later on any 'apt' call in > 'buildchroot' and 'image' for this configuration. > > I was wrong with kernel version, it's 3.16 - the standard Jessie > kernel. So for now, I'd like to drop this patch and add supported > hosts matrix to Isar. This workaround could be described here. I guess one could make such a workaround depending on "apt-get --version", to prevent it from causing undesired things in versions it is not for. Henning > Alex > > > > >> We can't require Isar users to always have up-to-dated host > >> systems. If this patch makes all Isar users with old kernels > >> happy, it's probably good for Isar in general :-) > > > > Sure, but there is a cost ... see previous point. > > > >>> > >>> If this patch needs to go in, it is missing the cleanup step. > >>> > >> > >> Sorry, what do you mean here by cleanup step? > > > > Sorry, misread the patch. I thought it placed a file in the rootfs. > > > > Henning > > > >> Alex > >> > >>> Henning > >>> > >>>>> Signed-off-by: Alexander Smirnov > >>>>> --- > >>>>> meta/recipes-core/isar-bootstrap/isar-bootstrap.bb | 4 +++- > >>>>> 1 file changed, 3 insertions(+), 1 deletion(-) > >>>>> > >>>>> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb > >>>>> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb index > >>>>> a38dd88..4cdefaa 100644 --- > >>>>> a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb +++ > >>>>> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb @@ -187,7 > >>>>> +187,9 @@ do_apt_update() { > >>>>> E="${@bb.utils.export_proxies(d)}" > >>>>> export DEBIAN_FRONTEND=noninteractive > >>>>> - sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y > >>>>> + sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y \ > >>>>> + -o > >>>>> APT::Cache-Start=50331648 + > >>>>> sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get > >>>>> dist-upgrade -y \ -o Debug::pkgProblemResolver=yes > >>>>> } > >>>>> > >>>> > >>> > >> > > >