Re: [PATCH] isar-bootstrap: debootstrap --include flag to support https:// sources

[Claudius Heine <claudius.heine.ext@siemens.com>]

Hi,

On 04/12/2018 14.35, Hosgor, Tolga (CT RDA DS EU TR MTS) wrote:
> Building 'isar-bootstrap-target' fails when there are HTTPS URIs in
> the distro APT sources and the first URI is not HTTPS.
> The first URI in the APT sources is passed to 'debootstrap' and
> the distro APT sources file is written to the isar bootstrap rootfs.
> Then, following 'apt-get update' fails due to apt-transport-https,
> ca-certificates being missing. In a normal workflow, 'debootstrap'
> automatically includes them if the MIRROR is HTTPS.
> 
> Signed-off-by: Hosgor, Tolga (CT RDA DS EU TR MTS) <tolga.hosgor@siemens.com>
> ---
>   .../isar-bootstrap/isar-bootstrap.inc         | 28 +++++++++++++++++--
>   1 file changed, 25 insertions(+), 3 deletions(-)
> 
> diff --git a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> index cfad136..33ad8e8 100644
> --- a/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> +++ b/meta/recipes-core/isar-bootstrap/isar-bootstrap.inc
> @@ -109,11 +109,15 @@ def aggregate_aptsources_list(d, file_list, file_out):
>                       out_fd.write("\n".encode())
>               out_fd.write("\n".encode())
>   
> -def get_distro_primary_source_entry(d, is_host=False):
> +def get_aptsources_list(d, is_host=False):
>       if is_host:
>           apt_sources_list = (d.getVar("HOST_DISTRO_APT_SOURCES", True) or "").split()
>       else:
>           apt_sources_list = (d.getVar("DISTRO_APT_SOURCES", True) or "").split()
> +    return apt_sources_list
> +
> +def get_distro_primary_source_entry(d, is_host=False):
> +    apt_sources_list = get_aptsources_list(d, is_host)
>       for entry in apt_sources_list:
>           entry_real = bb.parse.resolve_file(entry, d)
>           with open(entry_real, "r") as in_fd:
> @@ -125,6 +129,18 @@ def get_distro_primary_source_entry(d, is_host=False):
>                           return parsed[2:]
>       return ["", "", ""]
>   
> +def get_distro_have_https_source(d, is_host=False):
> +    for entry in get_aptsources_list(d, is_host):
> +        entry_real = bb.parse.resolve_file(entry, d)
> +        with open(entry_real, "r") as in_fd:
> +            for line in in_fd:
> +                parsed = parse_aptsources_list_line(line)
> +                if parsed:
> +                    parsed = get_apt_source_mirror(d, parsed)
> +                    if parsed[2].startswith("https://"):
> +                        return True
> +    return False

Maybe you can use the any [1] statement with a generator, to simplify 
the code here.



[1] https://docs.python.org/3/library/functions.html#any

> +
>   def get_distro_source(d, is_host):
>       return get_distro_primary_source_entry(d, is_host)[0]
>   
> @@ -138,6 +154,12 @@ def get_distro_components_argument(d, is_host):
>       else:
>           return ""
>   
> +def get_debootstrap_includes(d, is_host=False):
> +    if get_distro_have_https_source(d, is_host):
> +        return "locales,apt-transport-https,ca-certificates"

Those package names should be configured in the distro config.

Claudius

> +    else:
> +        return "locales"
> +
>   do_generate_keyring[dirs] = "${DL_DIR}"
>   do_generate_keyring[vardeps] += "DISTRO_APT_KEYS"
>   do_generate_keyring() {
> @@ -183,7 +205,7 @@ isar_bootstrap() {
>               if [ ${IS_HOST} ]; then
>                   ${DEBOOTSTRAP} --verbose \
>                                  --variant=minbase \
> -                               --include=locales \
> +                               --include="${@get_debootstrap_includes(d, True)}" \
>                                  ${@get_distro_components_argument(d, True)} \
>                                  ${DEBOOTSTRAP_KEYRING} \
>                                  "${@get_distro_suite(d, True)}" \
> @@ -194,7 +216,7 @@ isar_bootstrap() {
>                    "${DEBOOTSTRAP}" --verbose \
>                                     --variant=minbase \
>                                     --arch="${DISTRO_ARCH}" \
> -                                  --include=locales \
> +                                  --include="${@get_debootstrap_includes(d, False)}" \
>                                     ${@get_distro_components_argument(d, False)} \
>                                     ${DEBOOTSTRAP_KEYRING} \
>                                     "${@get_distro_suite(d, False)}" \
> 

-- 
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de