[PATCH 1/1] add var to configure apt rate limits

[PATCH 1/1] add var to configure apt rate limits

['Felix Moessbauer' via isar-users]

This patch introduces the variable ISAR_APT_DL_LIMIT which is used to
control the rate limit of apt.

Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
---
 RECIPE-API-CHANGELOG.md     | 5 +++++
 doc/user_manual.md          | 1 +
 meta/classes/rootfs.bbclass | 4 ++++
 3 files changed, 10 insertions(+)

diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
index 12ea93ec..65cd98b3 100644
--- a/RECIPE-API-CHANGELOG.md
+++ b/RECIPE-API-CHANGELOG.md
@@ -629,3 +629,8 @@ into kernel kbuild package.
     Only the "host" specific package is built automatically at cross builds.
 
   * Support emulated module build with cross-compiled kernel for linux-module
+
+### Rate-Limit apt fetching
+
+When downloading from debian snapshot mirrors, rate limits might apply.
+To limit the amount of parallel fetching to n kB / s, you can set `ISAR_APT_DL_LIMIT="<n>`.
diff --git a/doc/user_manual.md b/doc/user_manual.md
index 776ae52c..22d39073 100644
--- a/doc/user_manual.md
+++ b/doc/user_manual.md
@@ -433,6 +433,7 @@ Some other variables include:
  - `HOST_DISTRO_BOOTSTRAP_KEYS` - Analogously to DISTRO_BOOTSTRAP_KEYS: List of gpg key URIs used to verify apt bootstrap repo for the host.
  - `DISTRO_APT_PREMIRRORS` - The preferred mirror (append it to the default URI in the format `ftp.debian.org my.preferred.mirror`. This variable is optional. PREMIRRORS will be used only for the build. The final images will have the sources list as mentioned in DISTRO_APT_SOURCES.
  - `ISAR_USE_APT_SNAPSHOT` - Use a frozen apt snapshot instead of the live mirror. Optional.
+   `ISAR_APT_DL_LIMIT` - Rate limit the apt fetching to n kB / s. Optional.
  - `DISTRO_APT_SNAPSHOT_PREMIRROR` - Similar to `DISTRO_APT_PREMIRRORS` but for a snapshot, pre-defined for supported distros.
  - `ISAR_APT_SNAPSHOT_TIMESTAMP` - Timestamp of the apt snapshot. Automatically derived from `SOURCE_DATE_EPOCH` if not overwritten.
  - `THIRD_PARTY_APT_KEYS` - List of gpg key URIs used to verify apt repos for apt installation after bootstrapping.
diff --git a/meta/classes/rootfs.bbclass b/meta/classes/rootfs.bbclass
index 2e091e0c..fdb25eaa 100644
--- a/meta/classes/rootfs.bbclass
+++ b/meta/classes/rootfs.bbclass
@@ -111,6 +111,10 @@ rootfs_configure_apt() {
     mkdir -p '${ROOTFSDIR}/etc/apt/apt.conf.d'
     {
         echo 'Acquire::Retries "3";'
+        if [ -n "${ISAR_APT_DL_LIMIT}" ]; then
+            echo 'Acquire::http::Dl-Limit "${ISAR_APT_DL_LIMIT}";'
+            echo 'Acquire::https::Dl-Limit "${ISAR_APT_DL_LIMIT}";'
+        fi
         echo 'APT::Install-Recommends "0";'
         echo 'APT::Install-Suggests "0";'
     } > '${ROOTFSDIR}/etc/apt/apt.conf.d/50isar'
-- 
2.39.2

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/20240722103917.628994-1-felix.moessbauer%40siemens.com.

Re: [PATCH 1/1] add var to configure apt rate limits

[Uladzimir Bely]

On Mon, 2024-07-22 at 12:39 +0200, 'Felix Moessbauer' via isar-users
wrote:
> This patch introduces the variable ISAR_APT_DL_LIMIT which is used to
> control the rate limit of apt.
> 
> Signed-off-by: Felix Moessbauer <felix.moessbauer@siemens.com>
> ---
>  RECIPE-API-CHANGELOG.md     | 5 +++++
>  doc/user_manual.md          | 1 +
>  meta/classes/rootfs.bbclass | 4 ++++
>  3 files changed, 10 insertions(+)
> 
> diff --git a/RECIPE-API-CHANGELOG.md b/RECIPE-API-CHANGELOG.md
> index 12ea93ec..65cd98b3 100644
> --- a/RECIPE-API-CHANGELOG.md
> +++ b/RECIPE-API-CHANGELOG.md
> @@ -629,3 +629,8 @@ into kernel kbuild package.
>      Only the "host" specific package is built automatically at cross
> builds.
>  
>    * Support emulated module build with cross-compiled kernel for
> linux-module
> +
> +### Rate-Limit apt fetching
> +
> +When downloading from debian snapshot mirrors, rate limits might
> apply.
> +To limit the amount of parallel fetching to n kB / s, you can set
> `ISAR_APT_DL_LIMIT="<n>`.
> diff --git a/doc/user_manual.md b/doc/user_manual.md
> index 776ae52c..22d39073 100644
> --- a/doc/user_manual.md
> +++ b/doc/user_manual.md
> @@ -433,6 +433,7 @@ Some other variables include:
>   - `HOST_DISTRO_BOOTSTRAP_KEYS` - Analogously to
> DISTRO_BOOTSTRAP_KEYS: List of gpg key URIs used to verify apt
> bootstrap repo for the host.
>   - `DISTRO_APT_PREMIRRORS` - The preferred mirror (append it to the
> default URI in the format `ftp.debian.org my.preferred.mirror`. This
> variable is optional. PREMIRRORS will be used only for the build. The
> final images will have the sources list as mentioned in
> DISTRO_APT_SOURCES.
>   - `ISAR_USE_APT_SNAPSHOT` - Use a frozen apt snapshot instead of
> the live mirror. Optional.
> +   `ISAR_APT_DL_LIMIT` - Rate limit the apt fetching to n kB / s.
> Optional.
>   - `DISTRO_APT_SNAPSHOT_PREMIRROR` - Similar to
> `DISTRO_APT_PREMIRRORS` but for a snapshot, pre-defined for supported
> distros.
>   - `ISAR_APT_SNAPSHOT_TIMESTAMP` - Timestamp of the apt snapshot.
> Automatically derived from `SOURCE_DATE_EPOCH` if not overwritten.
>   - `THIRD_PARTY_APT_KEYS` - List of gpg key URIs used to verify apt
> repos for apt installation after bootstrapping.
> diff --git a/meta/classes/rootfs.bbclass
> b/meta/classes/rootfs.bbclass
> index 2e091e0c..fdb25eaa 100644
> --- a/meta/classes/rootfs.bbclass
> +++ b/meta/classes/rootfs.bbclass
> @@ -111,6 +111,10 @@ rootfs_configure_apt() {
>      mkdir -p '${ROOTFSDIR}/etc/apt/apt.conf.d'
>      {
>          echo 'Acquire::Retries "3";'
> +        if [ -n "${ISAR_APT_DL_LIMIT}" ]; then
> +            echo 'Acquire::http::Dl-Limit "${ISAR_APT_DL_LIMIT}";'
> +            echo 'Acquire::https::Dl-Limit "${ISAR_APT_DL_LIMIT}";'
> +        fi
>          echo 'APT::Install-Recommends "0";'
>          echo 'APT::Install-Suggests "0";'
>      } > '${ROOTFSDIR}/etc/apt/apt.conf.d/50isar'
> -- 
> 2.39.2
> 

Applied to next, thanks.

-- 
Best regards,
Uladzimir.



-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/isar-users/2c45de537a45033f6fb9a5669f4cabbd2bf25600.camel%40ilbers.de.