[RFC PATCH] classes/deb-dl-dir.bbclass: Download src with retry mechanism

[RFC PATCH] classes/deb-dl-dir.bbclass: Download src with retry mechanism

[Srinuvasan A]

From: Srinuvasan A <srinuvasan_a@mentor.com>

Sometime the source packages are not downloaded successfully due to
the low speed network, hence the builds get fails with timeout issue

Added a mechanism to attempt multiple try till the download get success.

Signed-off-by: Srinuvasan A <srinuvasan_a@mentor.com>
---
 meta/classes/deb-dl-dir.bbclass | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)

diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass
index 29bf45b..e335feb 100644
--- a/meta/classes/deb-dl-dir.bbclass
+++ b/meta/classes/deb-dl-dir.bbclass
@@ -55,7 +55,14 @@ debsrc_download() {
         [ -n "$dscfile" ] && continue
 
         sudo -E chroot --userspec=$( id -u ):$( id -g ) ${rootfs} \
-            sh -c ' mkdir -p "/deb-src/${1}/${2}" && cd "/deb-src/${1}/${2}" && apt-get -y --download-only --only-source source "$2"="$3" ' download-src "${rootfs_distro}" "${src}" "${version}"
+            sh -c ' mkdir -p "/deb-src/${1}/${2}" && cd "/deb-src/${1}/${2}" &&
+            download_src="apt-get -y --download-only --only-source source "$2"="$3""
+            download_status=1
+            while[ $download_status != 0 ]
+            do
+                $download_src
+                download_status=$?
+            done ' download-src "${rootfs_distro}" "${src}" "${version}"
     done
     ) 9>"${DEBSRCDIR}/${rootfs_distro}.lock"
 
-- 
2.25.1

Re: [RFC PATCH] classes/deb-dl-dir.bbclass: Download src with retry mechanism

[Jan Kiszka]

On 23.11.21 11:20, Srinuvasan A wrote:
> From: Srinuvasan A <srinuvasan_a@mentor.com>
> 
> Sometime the source packages are not downloaded successfully due to
> the low speed network, hence the builds get fails with timeout issue
> 
> Added a mechanism to attempt multiple try till the download get success.
> 
> Signed-off-by: Srinuvasan A <srinuvasan_a@mentor.com>
> ---
>  meta/classes/deb-dl-dir.bbclass | 9 ++++++++-
>  1 file changed, 8 insertions(+), 1 deletion(-)
> 
> diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass
> index 29bf45b..e335feb 100644
> --- a/meta/classes/deb-dl-dir.bbclass
> +++ b/meta/classes/deb-dl-dir.bbclass
> @@ -55,7 +55,14 @@ debsrc_download() {
>          [ -n "$dscfile" ] && continue
>  
>          sudo -E chroot --userspec=$( id -u ):$( id -g ) ${rootfs} \
> -            sh -c ' mkdir -p "/deb-src/${1}/${2}" && cd "/deb-src/${1}/${2}" && apt-get -y --download-only --only-source source "$2"="$3" ' download-src "${rootfs_distro}" "${src}" "${version}"
> +            sh -c ' mkdir -p "/deb-src/${1}/${2}" && cd "/deb-src/${1}/${2}" &&
> +            download_src="apt-get -y --download-only --only-source source "$2"="$3""
> +            download_status=1
> +            while[ $download_status != 0 ]
> +            do
> +                $download_src
> +                download_status=$?
> +            done ' download-src "${rootfs_distro}" "${src}" "${version}"
>      done
>      ) 9>"${DEBSRCDIR}/${rootfs_distro}.lock"
>  
> 

We had that discussion in the past, see
https://groups.google.com/d/msgid/isar-users/1d96541b-1263-773c-d340-35b0c73cb257%40siemens.com,
and that spot was address with a finite loop. Same should be done here.

But note that this is far from complete, see that thread above. Worst
issues: debootstrap has no option to enable a retry.

Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux