From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6824111115797004288 X-Received: by 2002:a05:651c:50e:: with SMTP id o14mr741647ljp.52.1588921624195; Fri, 08 May 2020 00:07:04 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a19:4285:: with SMTP id p127ls91070lfa.10.gmail; Fri, 08 May 2020 00:07:03 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwyVDsFd8HxXuZ+aV7x0Wut1TyzTwuUALr+OoC0pWkwAg50vfpUW8ynM5iy47ZxnAop//rt X-Received: by 2002:a05:6512:202:: with SMTP id a2mr925175lfo.42.1588921623407; Fri, 08 May 2020 00:07:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1588921623; cv=none; d=google.com; s=arc-20160816; b=o43qBduHSYnWHtb28aRQKbNW8LA44H4ZKm7v97msYBw6PtPtbHtaMrr44oC5X8SLIu vHDXEN5vixCbyE649GmDz/wt2XSKUjW4GXjiCIbBVLk3LtVDUz43xh4qyJyfmLObUaNj S4VPYEjKiIGa/I79MTOSy7VySnjpuK+42SF63JgfmR1B7hP5N48I9DENVqUT9YJDnnLs IqBsSl+d6QDJAMuADNqFzsY0RiQBJYRBRkA8IBkwwo+ssTN81vW12RbaOMOzhMRUQoN1 E/2G3R0xgzbYuu4iF3MtMBOBn9U+ua8UMKLYQTLP6663WDg4MskXuZLAGipzemt3W/gt GchQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:subject:cc:to:from:date; bh=/rqRatz9hTa1apSeQTjZ3joznD3ER+G5FTPlfpa4FRA=; b=COWIvFRCCGM/T+uLhtDtYlsrj1gJfgxrgYyDEheSN2lFi5maUzJl6wWN/zhjjHSdC5 MXMsx2J3E4+jChK25LO61qpJ1/k+4q7zwWA/p9ECGwq7zh383ZxKv9GXL+oHDzx+QITw FQgl87XtYlhUjsaOskZMw3M+HqO8FcN6nyJ7sQiRbkxrX3n4OwaJ5ntlaMnhVna9V4Mf XxjTvM5ONm2sUwnoJvcRc0wFTvAmH5A9aBKwYGp/2EzLbm6VaWzbYCkq86Yc5ZUJuBHB 7+/l8qw3ZvcChbRPseuwzm/z2dLn74yYzBkfGCl0SrzH1SoIN7jGBBkvFPtLGSHOKMSn Jw5w== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id q24si64407ljg.4.2020.05.08.00.07.02 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Fri, 08 May 2020 00:07:03 -0700 (PDT) Received-SPF: pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of henning.schild@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=henning.schild@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id 048771ne012032 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Fri, 8 May 2020 09:07:02 +0200 Received: from md1za8fc.ad001.siemens.net ([139.22.34.114]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 0487702T025237; Fri, 8 May 2020 09:07:01 +0200 Date: Fri, 8 May 2020 09:07:00 +0200 From: Henning Schild To: Jan Kiszka Cc: isar-users Subject: Re: [RFC][PATCH] rootfs: Retry downloads 3 times Message-ID: <20200508090700.17b19477@md1za8fc.ad001.siemens.net> In-Reply-To: References: <1d96541b-1263-773c-d340-35b0c73cb257@siemens.com> <20200507201357.5d24f91b@md1za8fc.ad001.siemens.net> X-Mailer: Claws Mail 3.17.5 (GTK+ 2.24.32; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit X-TUID: BhknxX1Gmll1 Am Fri, 8 May 2020 07:57:34 +0200 schrieb Jan Kiszka : > On 07.05.20 20:13, Henning Schild wrote: > > The idea is good but the patch is incomplete. > > > > To find all places where we download you might want to look at > > bitbake fetchers in general - do they have a retry count? > > > > Is there a variable controlling that? > > As explained in the commit log, it's not consistent in bitbake. At > least wget has such a retry counter. In other cases, bitbake seems to > quickly fall back to mirrors and fail. Ok now i know how to read that. > > > > After that look at all debian downloaders. You will find them when > > looking for "download-only", "apt-get update" any maybe more. They > > should use the central bitbake count or switch if available. > > > > Your patch found one of many "download-only" and ignores bitbake. > > Might be the most likely ... but we should try to be consistent. > > Yes, we likely also need this when fetching build deps. All images > are covered this way already, I also checked debootstrap but found no > retry switch there. So the result will remain "best effort" in > general. Therefore, I was also wondering if it's worth it. I do not think so. We do have the cache now, maybe we can deb_dl_dir_export after failed "download-only". That way you can at least retry on the outside. So interactive incremental builds should work on a shaky connection. In a later step we could introduce the arg for apt and write a loop around debian fetchers that can not retry ... debootstrap. That could help reduce the number of manual increments, and maybe help non-interactive builds with shaky networks. Henning > Background was may lengthy attempt yesterday to install a complex > image from snapshot.debian.org. The "solution" (that avoided a first > complete local mirror) was to throttle the download bandwidth to some > 1 MBit/s. Maybe there was just one bad mirror on the other side, but > it constantly killed the image installed. Retrying was a futile > attempt there, but I've also seen (few) nightly builds in the past > that stumbled over individual packet downloads where this might have > helped. > > Jan > > > > > Henning > > > > Am Thu, 7 May 2020 16:36:18 +0200 > > schrieb Jan Kiszka : > > > >> From: Jan Kiszka > >> > >> Avoids failing a complete rootfs installation in case of a short > >> hick-up. > >> > >> Signed-off-by: Jan Kiszka > >> --- > >> > >> Does something like this make sense? Do we have more places? > >> We have a retry of 2 on wget bitbake fetches, e.g. (via > >> FETCHCMD_wget). > >> > >> meta/classes/rootfs.bbclass | 2 +- > >> 1 file changed, 1 insertion(+), 1 deletion(-) > >> > >> diff --git a/meta/classes/rootfs.bbclass > >> b/meta/classes/rootfs.bbclass index 806e824c..afec1cbc 100644 > >> --- a/meta/classes/rootfs.bbclass > >> +++ b/meta/classes/rootfs.bbclass > >> @@ -123,7 +123,7 @@ rootfs_install_pkgs_download[weight] = "600" > >> rootfs_install_pkgs_download[isar-apt-lock] = "release-after" > >> rootfs_install_pkgs_download() { > >> sudo -E chroot '${ROOTFSDIR}' \ > >> - /usr/bin/apt-get ${ROOTFS_APT_ARGS} --download-only > >> ${ROOTFS_PACKAGES} > >> + /usr/bin/apt-get ${ROOTFS_APT_ARGS} -o Acquire::Retries=3 > >> --download-only ${ROOTFS_PACKAGES} } > >> > >> ROOTFS_INSTALL_COMMAND_BEFORE_EXPORT ??= "" > > > >