public inbox for isar-users@googlegroups.com
 help / color / mirror / Atom feed
From: Alexander Smirnov <asmirnov@ilbers.de>
To: Henning Schild <henning.schild@siemens.com>
Cc: isar-users <isar-users@googlegroups.com>
Subject: Re: [PATCH] isar-bootstrap: Increase cache room
Date: Tue, 8 May 2018 10:56:55 +0300	[thread overview]
Message-ID: <6fbef55a-276d-43ca-3039-8556c4b1ad8c@ilbers.de> (raw)
In-Reply-To: <20180507191213.7b68192b@md1pvb1c.ad001.siemens.net>

On 05/07/2018 08:12 PM, Henning Schild wrote:
> Am Mon, 7 May 2018 19:48:16 +0300
> schrieb Alexander Smirnov <asmirnov@ilbers.de>:
> 
>> Hi,
>>
>> On 05/07/2018 07:31 PM, Henning Schild wrote:
>>> Am Mon, 7 May 2018 18:57:36 +0300
>>> schrieb Alexander Smirnov <asmirnov@ilbers.de>:
>>>    
>>>> On 05/07/2018 06:48 PM, Alexander Smirnov wrote:
>>>>> According to the man page for 'apt.conf', there are three
>>>>> variables:
>>>>>     - Cache-Start: initial size of mmap cache room
>>>>>     - Cache-Grow: gap for dynamic mmap cache growth
>>>>>     - Cache-Limit: maximal cache size after growth
>>>>>
>>>>> If dynamic resize of mmap region is not avaialbe, the apt uses
>>>>> pre-allocated Cache-Start room [1] for mmap file.
>>>>>
>>>>> Building Isar on one of the Debian host with kernel 3.4, I got the
>>>>> following problem for 'qemuarm64-stretch' configuration:
>>>>>
>>>>> 8<--
>>>>> Hit:1 http://security.debian.org stretch/updates InRelease
>>>>> Ign:2 http://ftp.de.debian.org/debian stretch InRelease
>>>>> Hit:3 http://ftp.de.debian.org/debian stretch-updates InRelease
>>>>> Hit:4 http://ftp.de.debian.org/debian stretch Release
>>>>> E: Dynamic MMap ran out of room. Please increase the size of
>>>>> APT::Cache-Start. Current value: 25165824. (man 5 apt.conf) qemu:
>>>>> uncaught target signal 11 (Segmentation fault) - core dumped
>>>>> Segmentation fault 8<--
>>>>>
>>>>> I have no information, why exactly the room could not be re-sized
>>>>> on this system, but it would be better to increase the initial
>>>>> room size for apt. This patch increases the default apt cache
>>>>> twice.
>>>>
>>>> 1: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=657085
>>>>
>>>> Missed :-(
>>>
>>> What is your conclusion from that link and what does it mean for the
>>> patch?
>>>    
>>
>> This just the information I found about dynamic cache increasing.
>> Sometimes it could not work. Another fact is that increasing of
>> Cache-Limit doesn't help, so seems that mremap really doesn't work.
>>
>>> I do not like adding magic numbers and changing the default config,
>>> especially if the whole reason is to work around a problem only
>>> found on outdated build hosts (3.4 really?)
>>
>> I'm not 100% sure that kernel 3.4 is the source of the problem. Tried
>> to investigate, but for now without results.
>>
>> But if you try to google line:
>>
>>     E: Dynamic MMap ran out of room. Please increase the size of
>> APT::Cache-Start
>>
>> There will be tons of search results, so such problem occurred not
>> only on my host.
>>
>>
>> The man page says:
>>
>> ...on systems with a lot of configured sources it should be
>> increased...
>>
>> so for me it makes sense to increase this room even if there is only
>> one problem report. It doesn't break anything.
> 
> Oh yes it breaks the fact that we do not need to care about the value.
> If you hardcode one today you might be setting half the default after
> the next apt-update. ... because it is a magic number and not
> 
> set(get()*2)

Unfortunately this option is not get-able. It's defined as member of 
class in source code, which is initialized on class creation. There is 
no getter function to request this variable later.

8<--
apt-pkg/pkgcachegen.cc:

map_filesize_t const MapStart = _config->FindI("APT::Cache-Start", 
24*1024*1024);
...
return new DynamicMMap(*CacheF, Flags, MapStart, MapGrow, MapLimit);
8<--

Anyway, I noticed that build fails later on any 'apt' call in 
'buildchroot' and 'image' for this configuration.

I was wrong with kernel version, it's 3.16 - the standard Jessie kernel. 
So for now, I'd like to drop this patch and add supported hosts matrix 
to Isar. This workaround could be described here.

Alex

> 
>> We can't require Isar users to always have up-to-dated host systems.
>> If this patch makes all Isar users with old kernels happy, it's
>> probably good for Isar in general :-)
> 
> Sure, but there is a cost ... see previous point.
>   
>>>
>>> If this patch needs to go in, it is missing the cleanup step.
>>>    
>>
>> Sorry, what do you mean here by cleanup step?
> 
> Sorry, misread the patch. I thought it placed a file in the rootfs.
> 
> Henning
> 
>> Alex
>>
>>> Henning
>>>    
>>>>> Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
>>>>> ---
>>>>>     meta/recipes-core/isar-bootstrap/isar-bootstrap.bb | 4 +++-
>>>>>     1 file changed, 3 insertions(+), 1 deletion(-)
>>>>>
>>>>> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
>>>>> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb index
>>>>> a38dd88..4cdefaa 100644 ---
>>>>> a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb +++
>>>>> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb @@ -187,7
>>>>> +187,9 @@ do_apt_update() {
>>>>>         E="${@bb.utils.export_proxies(d)}"
>>>>>         export DEBIAN_FRONTEND=noninteractive
>>>>> -    sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y
>>>>> +    sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y \
>>>>> +                                      -o
>>>>> APT::Cache-Start=50331648 +
>>>>>         sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade
>>>>> -y \ -o Debug::pkgProblemResolver=yes
>>>>>     }
>>>>>       
>>>>   
>>>    
>>
> 

-- 
With best regards,
Alexander Smirnov

ilbers GmbH
Baierbrunner Str. 28c
D-81379 Munich
+49 (89) 122 67 24-0
http://ilbers.de/
Commercial register Munich, HRB 214197
General manager: Baurzhan Ismagulov

  reply	other threads:[~2018-05-08  7:57 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-05-07 15:48 Alexander Smirnov
2018-05-07 15:57 ` Alexander Smirnov
2018-05-07 16:31   ` Henning Schild
2018-05-07 16:48     ` Alexander Smirnov
2018-05-07 17:12       ` Henning Schild
2018-05-08  7:56         ` Alexander Smirnov [this message]
2018-05-08  8:56           ` Henning Schild
2018-05-08  9:07             ` Alexander Smirnov
2018-05-08  9:22               ` Henning Schild
2018-05-08  9:27                 ` Alexander Smirnov
2018-05-08 10:30                   ` Henning Schild
2018-05-14  8:54 ` Claudius Heine
2018-05-14  9:11   ` Claudius Heine
2018-05-14  9:21     ` Alexander Smirnov

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=6fbef55a-276d-43ca-3039-8556c4b1ad8c@ilbers.de \
    --to=asmirnov@ilbers.de \
    --cc=henning.schild@siemens.com \
    --cc=isar-users@googlegroups.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox