Re: [PATCH 1/1] deb-dl-dir: fix package source download

["'Niedermayr, BENEDIKT' via isar-users" <isar-users@googlegroups.com>]

On 22.01.25 15:00, Kiszka, Jan (FT RPD CED) wrote:
> On 22.01.25 14:49, 'Benedikt Niedermayr' via isar-users wrote:
>> Some of the packages that are part of the debootstrap base filesystem
> Just "bootstrap", at least as long as we still have debootstrap as
> legacy option beside mmdebstrap.
>
> Is this issues possibly a fallout from the mmdebstrap migration?

Hi Jan,

just tested it running debootstrap(PREFERRED_PROVIDER_bootstrap-*).

Using debootstrap will download all packages. Seems that mmdebstrap 
introduced some regression here.

Are there any known issues using "dpkg-query" in terms of multiconfig? I 
would like to avoid backing up and restoring

the dpkg.log file which feels more like a workaround compared to using 
dpkg-query.


>
> Jan
>
>> were not downloaded when activating "cache-deb-src" via
>> BASE_REPO_FEATURES.
>>
>> It seems that some (or all) packages are not listed in
>> "/var/log/dpkg.log" leading to skip the download of these files.
>>
>> Package built by using the dpkg-prebuilt.bbclass tend to reference
>> binary packages without providing any source package reference.
>> To handle these currently valid cases the download function simply
>> skips packages that could not be downloaded and prints a warning.
>>
>> Signed-off-by: Benedikt Niedermayr <benedikt.niedermayr@siemens.com>
>> ---
>>   meta/classes/deb-dl-dir.bbclass | 39 ++++++++++-----------------------
>>   1 file changed, 11 insertions(+), 28 deletions(-)
>>
>> diff --git a/meta/classes/deb-dl-dir.bbclass b/meta/classes/deb-dl-dir.bbclass
>> index 7ebd057e5504..9f6d205894dc 100644
>> --- a/meta/classes/deb-dl-dir.bbclass
>> +++ b/meta/classes/deb-dl-dir.bbclass
>> @@ -5,23 +5,6 @@
>>   
>>   inherit repository
>>   
>> -is_not_part_of_current_build() {
>> -    local package="$( dpkg-deb --show --showformat '${Package}' "${1}" )"
>> -    local arch="$( dpkg-deb --show --showformat '${Architecture}' "${1}" )"
>> -    local version="$( dpkg-deb --show --showformat '${Version}' "${1}" )"
>> -    # Since we are parsing all the debs in DEBDIR, we can to some extend
>> -    # try to eliminate some debs that are not part of the current multiconfig
>> -    # build using the below method.
>> -    local output="$( grep -xhs ".* status installed ${package}:${arch} ${version}" \
>> -            "${IMAGE_ROOTFS}"/var/log/dpkg.log \
>> -            "${SCHROOT_HOST_DIR}"/var/log/dpkg.log \
>> -            "${SCHROOT_TARGET_DIR}"/var/log/dpkg.log \
>> -            "${SCHROOT_HOST_DIR}"/tmp/dpkg_common.log \
>> -            "${SCHROOT_TARGET_DIR}"/tmp/dpkg_common.log | head -1 )"
>> -
>> -    [ -z "${output}" ]
>> -}
>> -
>>   debsrc_do_mounts() {
>>       sudo -s <<EOSUDO
>>       set -e
>> @@ -54,17 +37,17 @@ debsrc_download() {
>>       ( flock 9
>>       set -e
>>       printenv | grep -q BB_VERBOSE_LOGS && set -x
>> -    find "${rootfs}/var/cache/apt/archives/" -maxdepth 1 -type f -iname '*\.deb' | while read package; do
>> -        is_not_part_of_current_build "${package}" && continue
>> -        local src="$( dpkg-deb --show --showformat '${source:Package}' "${package}" )"
>> -        local version="$( dpkg-deb --show --showformat '${source:Version}' "${package}" )"
>> -        local dscname="$(echo ${src}_${version} | sed -e 's/_[0-9]\+:/_/')"
>> -        local dscfile=$(find "${DEBSRCDIR}"/"${rootfs_distro}" -name "${dscname}.dsc")
>> -        [ -n "$dscfile" ] && continue
>> -
>> -        sudo -E chroot --userspec=$( id -u ):$( id -g ) ${rootfs} \
>> -            sh -c ' mkdir -p "/deb-src/${1}/${2}" && cd "/deb-src/${1}/${2}" && apt-get -y --download-only --only-source source "$2"="$3" ' download-src "${rootfs_distro}" "${src}" "${version}"
>> -    done
>> +    sudo -E chroot --userspec=$( id -u ):$( id -g ) ${rootfs} \
>> +        dpkg-query -f '${source:Package} ${source:Version}\n' -W | while read -r src srcver; do
>> +            ver_stripped=$(echo "$srcver" | sed 's/^[0-9]*://')
>> +            test -f "${DEBSRCDIR}/${rootfs_distro}/${src}/${src}_${ver_stripped}.dsc" && continue
>> +
>> +            # Note: package built by using dpkg-prebuilt are tend to be missing
>> +            sudo -E chroot --userspec=$( id -u ):$( id -g ) ${rootfs} \
>> +                sh -c ' mkdir -p "/deb-src/${1}/${2}" && cd "/deb-src/${1}/${2}" && apt-get -y --download-only --only-source source "$2"="$3" ' \
>> +                    download-src "${rootfs_distro}" "${src}" "${srcver}" || \
>> +                    bbwarn "Failed to download source package ${src}_${srcver}"
>> +        done
>>       ) 9>"${DEBSRCDIR}/${rootfs_distro}.lock"
>>   
>>       debsrc_undo_mounts "${rootfs}"
>

-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com.
To view this discussion visit https://groups.google.com/d/msgid/isar-users/f47301a8-a16c-4969-8b5e-b376c5cc4dfb%40siemens.com.