Re: [PATCH] isar-bootstrap: Increase cache room

[Henning Schild <henning.schild@siemens.com>]

Am Mon, 7 May 2018 19:48:16 +0300
schrieb Alexander Smirnov <asmirnov@ilbers.de>:

> Hi,
> 
> On 05/07/2018 07:31 PM, Henning Schild wrote:
> > Am Mon, 7 May 2018 18:57:36 +0300
> > schrieb Alexander Smirnov <asmirnov@ilbers.de>:
> >   
> >> On 05/07/2018 06:48 PM, Alexander Smirnov wrote:  
> >>> According to the man page for 'apt.conf', there are three
> >>> variables:
> >>>    - Cache-Start: initial size of mmap cache room
> >>>    - Cache-Grow: gap for dynamic mmap cache growth
> >>>    - Cache-Limit: maximal cache size after growth
> >>>
> >>> If dynamic resize of mmap region is not avaialbe, the apt uses
> >>> pre-allocated Cache-Start room [1] for mmap file.
> >>>
> >>> Building Isar on one of the Debian host with kernel 3.4, I got the
> >>> following problem for 'qemuarm64-stretch' configuration:
> >>>
> >>> 8<--
> >>> Hit:1 http://security.debian.org stretch/updates InRelease
> >>> Ign:2 http://ftp.de.debian.org/debian stretch InRelease
> >>> Hit:3 http://ftp.de.debian.org/debian stretch-updates InRelease
> >>> Hit:4 http://ftp.de.debian.org/debian stretch Release
> >>> E: Dynamic MMap ran out of room. Please increase the size of
> >>> APT::Cache-Start. Current value: 25165824. (man 5 apt.conf) qemu:
> >>> uncaught target signal 11 (Segmentation fault) - core dumped
> >>> Segmentation fault 8<--
> >>>
> >>> I have no information, why exactly the room could not be re-sized
> >>> on this system, but it would be better to increase the initial
> >>> room size for apt. This patch increases the default apt cache
> >>> twice. 
> >>
> >> 1: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=657085
> >>
> >> Missed :-(  
> > 
> > What is your conclusion from that link and what does it mean for the
> > patch?
> >   
> 
> This just the information I found about dynamic cache increasing. 
> Sometimes it could not work. Another fact is that increasing of 
> Cache-Limit doesn't help, so seems that mremap really doesn't work.
> 
> > I do not like adding magic numbers and changing the default config,
> > especially if the whole reason is to work around a problem only
> > found on outdated build hosts (3.4 really?)  
> 
> I'm not 100% sure that kernel 3.4 is the source of the problem. Tried
> to investigate, but for now without results.
> 
> But if you try to google line:
> 
>    E: Dynamic MMap ran out of room. Please increase the size of 
> APT::Cache-Start
> 
> There will be tons of search results, so such problem occurred not
> only on my host.
> 
> 
> The man page says:
> 
> ...on systems with a lot of configured sources it should be
> increased...
> 
> so for me it makes sense to increase this room even if there is only
> one problem report. It doesn't break anything.

Oh yes it breaks the fact that we do not need to care about the value.
If you hardcode one today you might be setting half the default after
the next apt-update. ... because it is a magic number and not

set(get()*2)

> We can't require Isar users to always have up-to-dated host systems.
> If this patch makes all Isar users with old kernels happy, it's
> probably good for Isar in general :-)

Sure, but there is a cost ... see previous point.
 
> > 
> > If this patch needs to go in, it is missing the cleanup step.
> >   
> 
> Sorry, what do you mean here by cleanup step?

Sorry, misread the patch. I thought it placed a file in the rootfs.

Henning

> Alex
> 
> > Henning
> >   
> >>> Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
> >>> ---
> >>>    meta/recipes-core/isar-bootstrap/isar-bootstrap.bb | 4 +++-
> >>>    1 file changed, 3 insertions(+), 1 deletion(-)
> >>>
> >>> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb
> >>> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb index
> >>> a38dd88..4cdefaa 100644 ---
> >>> a/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb +++
> >>> b/meta/recipes-core/isar-bootstrap/isar-bootstrap.bb @@ -187,7
> >>> +187,9 @@ do_apt_update() {
> >>>        E="${@bb.utils.export_proxies(d)}"
> >>>        export DEBIAN_FRONTEND=noninteractive
> >>> -    sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y
> >>> +    sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get update -y \
> >>> +                                      -o
> >>> APT::Cache-Start=50331648 +
> >>>        sudo -E chroot "${ROOTFSDIR}" /usr/bin/apt-get dist-upgrade
> >>> -y \ -o Debug::pkgProblemResolver=yes
> >>>    }
> >>>      
> >>  
> >   
>