Re: [RFC][PATCH 2/3] classes/dsc: Basic Debian .dsc backend implementation

[Henning Schild <henning.schild@siemens.com>]

Am Mon, 23 Apr 2018 16:21:45 +0300
schrieb Alexander Smirnov <asmirnov@ilbers.de>:

> This patch introduces class to work with Debian .dsc files. The
> main goal is to simplify re-building and customization of original
> Debian packages.
> 
> To rebuild package from the upstream the following recipe template
> could be used:
> 
>   DSC_URI = "http://ftp.de.debian.org/...dsc;md5sum=..."
>   inherit debian-dsc
> 
> All the remaining information bitbake will derive automatically.
> This is only first implementation, so there are still several open
> issues like:
>  - Chose correct upstream package version depending on Debian distro
>  - quilt-3.0 source format support
>  - Cross-recipe dependencies recognition
> 
> Signed-off-by: Alexander Smirnov <asmirnov@ilbers.de>
> ---
>  meta/classes/debian-dsc.bbclass | 112
> ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 112
> insertions(+) create mode 100644 meta/classes/debian-dsc.bbclass
> 
> diff --git a/meta/classes/debian-dsc.bbclass
> b/meta/classes/debian-dsc.bbclass new file mode 100644
> index 0000000..4159e3a
> --- /dev/null
> +++ b/meta/classes/debian-dsc.bbclass
> @@ -0,0 +1,112 @@
> +# Debian .dsc backend
> +#
> +# This software is a part of ISAR.
> +# Copyright (c) ilbers GmbH, 2018
> +#
> +# SPDX-License-Identifier: MIT
> +
> +# List of non-standard variables added to package workspace
> +#
> +# Set by user:
> +#  * DSC_URI - uri link to .dsc file in upstream apt
> +#
> +# Generated automatically:
> +#  * DEBIAN_URI - uri link to Debian patch in upstream apt
> +
> +python __anonymous() {
> +    # Fetch .dsc package file
> +    dsc_uri = (d.getVar("DSC_URI", True) or "").split()
> +    if len(dsc_uri) == 0:
> +        return
> +
> +    try:
> +        fetcher = bb.fetch2.Fetch(dsc_uri, d)
> +        fetcher.download()
> +    except bb.fetch2.BBFetchException as e:
> +        raise bb.build.FuncFailed(e)
> +
> +    # Open .dsc file from downloads
> +    dl_dir = d.getVar("DL_DIR", True)
> +    dsc_file = (dsc_uri[0].split(";")[0]).split("/")[-1]
> +    filepath = dl_dir + "/" + dsc_file
> +
> +    pv = ""
> +    src_uri = ""
> +
> +    # Parse .dsc file to get package details
> +    with open(filepath, "r") as file:
> +        line = file.readline()
> +
> +        while line:
> +            # Get package version and export PV
> +            if line.startswith("Version:") and not pv:
> +                pv = line.split(": ")[-1].rstrip()
> +                d.setVar("PV", pv)
> +
> +            # Get package and Debian patch arcives names
> +            if line.startswith("Files:") and not src_uri:
> +                line = file.readline()
> +                src = line.split(" ")[-1].rstrip()
> +                src_uri = (dsc_uri[0].rsplit("/", 1))[0] + "/" + src
> +                src_uri = src_uri + ";md5sum=" + line.split("
> ")[1].rstrip()
> +                d.setVar("SRC_URI", src_uri)
> +
> +                line = file.readline()
> +                debian = line.split(" ")[-1].rstrip()
> +                debian_uri = (dsc_uri[0].rsplit("/", 1))[0] + "/" +
> debian
> +                debian_uri = debian_uri + ";md5sum=" + line.split("
> ")[1].rstrip()
> +                d.setVar("DEBIAN_URI", debian_uri)
> +
> +            line = file.readline()
> +
> +        file.close()
> +
> +    # Set correct path to unpacked sources
> +    pv_orig = (src.split("_")[-1]).split(".orig")[0]
> +    pn = d.getVar("PN", True)
> +    d.setVar("S", pn + "-" + pv_orig)
> +}
> +
> +python do_fetch_append() {
> +    debian_uri = (d.getVar("DEBIAN_URI", True) or "").split()
> +    if len(debian_uri) == 0:
> +        return
> +
> +    try:
> +        fetcher = bb.fetch2.Fetch(debian_uri, d)
> +        fetcher.download()
> +    except bb.fetch2.BBFetchException as e:
> +        raise bb.build.FuncFailed(e)
> +}
> +
> +python do_unpack_append() {
> +    import subprocess
> +
> +    s = d.getVar("S", True)
> +    workdir = d.getVar("WORKDIR", True)
> +    debian_uri = (d.getVar("DEBIAN_URI", True) or "").split()
> +
> +    try:
> +        fetcher = bb.fetch2.Fetch(debian_uri, d)
> +        fetcher.unpack(workdir + "/" + s)
> +    except bb.fetch2.BBFetchException as e:
> +        raise bb.build.FuncFailed(e)
> +}
> +
> +# NOTE: this is workaround to serve quilt-3.0 package format support.
> +#       To build such package, the original tarball should be placed
> +#       in "../" folder. In general, this should be handled by Isar
> +#       core build classes, but for now this class in the only one
> user +#       for quilt-3.0, so for the first implementation let"s
> keep it here. +# Open issues:
> +#  - How to identify if package has quilt-3.0 source format?

Had the same issue... The format is in the .dsc. We should check which
other formats change what we need to build.

> +#  - How to avoid copying of tarballs from downloads to working dir?

i fetched twice, once with unpack=false

Henning

> +do_build_prepend() {
> +    dl_dir="${@ d.getVar('DL_DIR', True)}"
> +    workdir="${@ d.getVar('WORKDIR', True)}"
> +    file="${@ ((d.getVar('SRC_URI',
> True)).split(";")[0]).split("/")[-1]}" +
> +    install -m 644 $dl_dir/$file $workdir
> +}
> +
> +inherit dpkg