From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6547633542169362432 X-Received: by 10.80.158.13 with SMTP id z13mr7241125ede.9.1524499881070; Mon, 23 Apr 2018 09:11:21 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 10.80.214.21 with SMTP id x21ls10284008edi.7.gmail; Mon, 23 Apr 2018 09:11:20 -0700 (PDT) X-Google-Smtp-Source: AIpwx4/bUWZFXM9BjZ13Vl48knw6SIBxNPm1Xgc0b9+HfqtGvmeuIUAtfmUiiJso9LW7l7fsWvn+ X-Received: by 10.80.242.145 with SMTP id f17mr4166762edm.11.1524499880461; Mon, 23 Apr 2018 09:11:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1524499880; cv=none; d=google.com; s=arc-20160816; b=Trq23uQ8J7GUCHC63XrWcm64dXEDzhAshOjlW78G3y39zuVVs2ZU70mlZIooC6cKcH jhbViRdgG5jTrycVI8eH7Y7STMjNR37+gp0ElX51rU3GayCvsSsujsQRijr4AqTKi5K+ Ezq/hDW63fkPr51FTwdWHYGm36Dsq0T76qw9cg4ufjZa41nI4CJxezZh9WTVHHRwbA0j L/3opNyxMHtCfnXqGkCVG5D/zbICyucnaeDlp10lOkpwjl7shnz+0xPIkfTAgUo0NMtx KKyWwet4HazZbVkq4tizNhFxkVnLfjvOEUJwT+B2WFKqkXFJWJzBfmaHKcjhD0CZ1pH3 kOXQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject :arc-authentication-results; bh=U0of+7tiuZdAi+KR9Qnl2tS3HAt3uMY0u3Jv3M7L3z8=; b=EnTsVDu7TlIFsLhB1T5SOzxlJIICWv4vsYbpCmIxRSMROH3ZQKj2b+4TQh7zd7mAkj xolaoWAWLjEUCDzTj1EjmeXXeqzIiI5ZIhDJ+UeY6qa0G+eO1kuVBJ4gq/vPU8ysELLA 5QsWNEWZIecKM6ZvxEZ9FmqdiNO4WsQlDopPq6cTDIZoQN0D7qDV6ZzZqsI9v5rCuugE Cbe5beO1DA9O0YBdo/QW0+63vETpQP/GfQqRwsEgBNbMDQQEqyYP2c3/ghXXqgpGgye/ 20v60olMkKu9fzGCNEfwWTyiu2QgcrnehAMehjfQ2wjzhIpZMt6z/fBObAQT9/BFs+CP JWIQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id v10si555745edf.4.2018.04.23.09.11.20 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 23 Apr 2018 09:11:20 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Received: from [10.0.2.15] ([188.227.110.165]) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id w3NGBH8F012975 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Mon, 23 Apr 2018 18:11:19 +0200 Subject: Re: [RFC][PATCH 2/3] classes/dsc: Basic Debian .dsc backend implementation To: Henning Schild Cc: isar-users@googlegroups.com References: <20180423132146.14743-1-asmirnov@ilbers.de> <20180423132146.14743-3-asmirnov@ilbers.de> <20180423170457.3e25e51a@mmd1pvb1c.ad001.siemens.net> From: Alexander Smirnov Message-ID: Date: Mon, 23 Apr 2018 19:11:12 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.4.0 MIME-Version: 1.0 In-Reply-To: <20180423170457.3e25e51a@mmd1pvb1c.ad001.siemens.net> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: SSzslmWEdiaX On 04/23/2018 06:04 PM, Henning Schild wrote: > Am Mon, 23 Apr 2018 16:21:45 +0300 > schrieb Alexander Smirnov : > >> This patch introduces class to work with Debian .dsc files. The >> main goal is to simplify re-building and customization of original >> Debian packages. >> >> To rebuild package from the upstream the following recipe template >> could be used: >> >> DSC_URI = "http://ftp.de.debian.org/...dsc;md5sum=..." >> inherit debian-dsc >> >> All the remaining information bitbake will derive automatically. >> This is only first implementation, so there are still several open >> issues like: >> - Chose correct upstream package version depending on Debian distro >> - quilt-3.0 source format support >> - Cross-recipe dependencies recognition >> >> Signed-off-by: Alexander Smirnov >> --- >> meta/classes/debian-dsc.bbclass | 112 >> ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 112 >> insertions(+) create mode 100644 meta/classes/debian-dsc.bbclass >> >> diff --git a/meta/classes/debian-dsc.bbclass >> b/meta/classes/debian-dsc.bbclass new file mode 100644 >> index 0000000..4159e3a >> --- /dev/null >> +++ b/meta/classes/debian-dsc.bbclass >> @@ -0,0 +1,112 @@ >> +# Debian .dsc backend >> +# >> +# This software is a part of ISAR. >> +# Copyright (c) ilbers GmbH, 2018 >> +# >> +# SPDX-License-Identifier: MIT >> + >> +# List of non-standard variables added to package workspace >> +# >> +# Set by user: >> +# * DSC_URI - uri link to .dsc file in upstream apt >> +# >> +# Generated automatically: >> +# * DEBIAN_URI - uri link to Debian patch in upstream apt >> + >> +python __anonymous() { >> + # Fetch .dsc package file >> + dsc_uri = (d.getVar("DSC_URI", True) or "").split() >> + if len(dsc_uri) == 0: >> + return > > Can we not use SRC_URI and append more sources as we read .dscs? A > copy of the fetching code just reminds me of the broken fetch/unpack in > the early Isar days. > Because i have got my dscs in tarballs and in git ... and i am behind a > proxy. Here I consider only traditional Debian way, when.dsc file is stored directly in apt with .orig tarball and .debian patch: http://ftp.de.debian.org/debian/pool/main/h/hello/ So this class intended to work with Debian apts. > >> + try: >> + fetcher = bb.fetch2.Fetch(dsc_uri, d) >> + fetcher.download() >> + except bb.fetch2.BBFetchException as e: >> + raise bb.build.FuncFailed(e) >> + >> + # Open .dsc file from downloads >> + dl_dir = d.getVar("DL_DIR", True) >> + dsc_file = (dsc_uri[0].split(";")[0]).split("/")[-1] >> + filepath = dl_dir + "/" + dsc_file >> + >> + pv = "" >> + src_uri = "" >> + >> + # Parse .dsc file to get package details >> + with open(filepath, "r") as file: >> + line = file.readline() >> + >> + while line: >> + # Get package version and export PV >> + if line.startswith("Version:") and not pv: >> + pv = line.split(": ")[-1].rstrip() >> + d.setVar("PV", pv) >> + >> + # Get package and Debian patch arcives names >> + if line.startswith("Files:") and not src_uri: >> + line = file.readline() >> + src = line.split(" ")[-1].rstrip() >> + src_uri = (dsc_uri[0].rsplit("/", 1))[0] + "/" + src >> + src_uri = src_uri + ";md5sum=" + line.split(" >> ")[1].rstrip() >> + d.setVar("SRC_URI", src_uri) >> + >> + line = file.readline() >> + debian = line.split(" ")[-1].rstrip() >> + debian_uri = (dsc_uri[0].rsplit("/", 1))[0] + "/" + >> debian >> + debian_uri = debian_uri + ";md5sum=" + line.split(" >> ")[1].rstrip() >> + d.setVar("DEBIAN_URI", debian_uri) >> + >> + line = file.readline() >> + >> + file.close() >> + >> + # Set correct path to unpacked sources >> + pv_orig = (src.split("_")[-1]).split(".orig")[0] >> + pn = d.getVar("PN", True) >> + d.setVar("S", pn + "-" + pv_orig) >> +} >> + >> +python do_fetch_append() { >> + debian_uri = (d.getVar("DEBIAN_URI", True) or "").split() >> + if len(debian_uri) == 0: >> + return >> + >> + try: >> + fetcher = bb.fetch2.Fetch(debian_uri, d) >> + fetcher.download() >> + except bb.fetch2.BBFetchException as e: >> + raise bb.build.FuncFailed(e) >> +} > > Wow another fetcher ... same question as above. Upstream Debian source package contains 3 parts: - .dsc - .orig.tar.xz - .debian.tar.xz So: - First you need to fetch .dsc to get information about remaining 2 artifacts. - .orig.tar.xz is fetched by standard fetcher. - .debian.tar.xz is fetched here. > >> +python do_unpack_append() { >> + import subprocess >> + >> + s = d.getVar("S", True) >> + workdir = d.getVar("WORKDIR", True) >> + debian_uri = (d.getVar("DEBIAN_URI", True) or "").split() >> + >> + try: >> + fetcher = bb.fetch2.Fetch(debian_uri, d) >> + fetcher.unpack(workdir + "/" + s) > > fetch^3 this time with unpack Again: - .orig.tar.xz is unpacked by standard fetcher - the content of .debian.tar.xz should be unpacked into the folder from the .orig.tar.xz archive So in general, this is an attempt to collect and prepare all the artifacts to make it usable with dpkg Isar class. Alex > > Henning > >> + except bb.fetch2.BBFetchException as e: >> + raise bb.build.FuncFailed(e) >> +} >> + >> +# NOTE: this is workaround to serve quilt-3.0 package format support. >> +# To build such package, the original tarball should be placed >> +# in "../" folder. In general, this should be handled by Isar >> +# core build classes, but for now this class in the only one >> user +# for quilt-3.0, so for the first implementation let"s >> keep it here. +# Open issues: >> +# - How to identify if package has quilt-3.0 source format? >> +# - How to avoid copying of tarballs from downloads to working dir? >> +do_build_prepend() { >> + dl_dir="${@ d.getVar('DL_DIR', True)}" >> + workdir="${@ d.getVar('WORKDIR', True)}" >> + file="${@ ((d.getVar('SRC_URI', >> True)).split(";")[0]).split("/")[-1]}" + >> + install -m 644 $dl_dir/$file $workdir >> +} >> + >> +inherit dpkg >