From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6655521619015892992 X-Received: by 2002:a2e:8593:: with SMTP id b19mr609777lji.5.1550252605690; Fri, 15 Feb 2019 09:43:25 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a2e:7e08:: with SMTP id z8-v6ls897927ljc.1.gmail; Fri, 15 Feb 2019 09:43:24 -0800 (PST) X-Google-Smtp-Source: AHgI3Ib/v+G2oF4dsGfdv4D7bDvUSTdCWrrEgHgKLzfWONXtARtI+d4T2tfByPuoca+ARcpiLCik X-Received: by 2002:a2e:9613:: with SMTP id v19mr633892ljh.16.1550252604932; Fri, 15 Feb 2019 09:43:24 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550252604; cv=none; d=google.com; s=arc-20160816; b=rtjoUXNtFYokyFEAeefB1gyuw9nSFXeh+/IggamuRRT8YsxAXdMuYREPIZon3pZKD2 AXiAz8HAIG+gQZ9TEQk/i7mIoKNiF+zwNgSChJdsm4u6/qIOEZnSokCYvsiPFI0COYbx Lbqsg3P538SnGH3vrlwg2pD5rO+24eQMCZPQjwUJUUzrVS09FtZ9bppE+6K7dEsd5jPp Owd6VLNSpegnejRh68WVljygj8rW9AsRb8qKLljmtwpPgPuvGuTnawkUuCsDgQeJjAoP aWINoXE+QOWJ7UkQeYwKPQZo5MTD10uVpYk0ZmDQhuuA2iC9RgP/8VmKfZV5W241116I gK2w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:organization:from:references:to:subject; bh=uTiEFaXVqF48we3wKFAZ181QSe9NjVal7jLqyAhN8J4=; b=BrGiqZ3u2rC/pEuQ2dSxfjuNN2YSGFyXdOO394nihRY2b2tsg92ozn4Kzd1Ld/ly1x todEe+/vcugaa6Jr4H08/IXDlRkeIHV8w2AAqn+vQr4JYckFvP3vykzp0aTiuMjNC1fn JoY+5z4+hobo+ptAHrnOs9ze/zCwkvC32I+oLqyOBl3ZbNLxnbO66yJJs2pWg9qTxtMM /NxYD5enywkEh7CnZIV6pzkVNgPlv9NTD5jYSWx8IE0rBTwvHFe+i3J2pFtrFHBad6++ mZJ+Vl//59WEuXASgCYWDHmPzqR02sqlS5OZGLRK1aqEJpVaZrihJ54mAmwzK+RAUChX EetA== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of mosipov@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=mosipov@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id w23si260486lfl.5.2019.02.15.09.43.24 for (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256); Fri, 15 Feb 2019 09:43:24 -0800 (PST) Received-SPF: pass (google.com: domain of mosipov@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of mosipov@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=mosipov@ilbers.de Received: from [192.168.50.163] (d51A48A80.access.telenet.be [81.164.138.128]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8) with ESMTPSA id x1FHhN3A001286 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Fri, 15 Feb 2019 18:43:23 +0100 Subject: Re: [PATCH v2] wic-img: import support for .wks.in files from oe-core To: Cedric Hombourger , isar-users@googlegroups.com References: <1549653186-1216-1-git-send-email-Cedric_Hombourger@mentor.com> From: "Maxim Yu. Osipov" Organization: ilbers GmbH Message-ID: Date: Fri, 15 Feb 2019 18:43:28 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.4.0 MIME-Version: 1.0 In-Reply-To: <1549653186-1216-1-git-send-email-Cedric_Hombourger@mentor.com> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: mCz8ERwn2PE1 On 2/8/19 8:13 PM, Cedric Hombourger wrote: > OpenEmbedded supports use of variables in wks files if they are > suffixed with .wks.in (instead of .wks). Use of this suffix adds > a task to the build process to expand variables to their values > when known or to the empty string when they are not. The result > of the evaluation is placed in a regular .wks file placed in the > build folder. > > The .wks file for the de0-nano-soc machine was modified to show > use of this feature and get it regularly tested as part of our > regular Isar builds. Applied to the 'next', Thanks, Maxim. > Signed-off-by: Cedric Hombourger > --- > doc/user_manual.md | 2 + > meta-isar/conf/machine/de0-nano-soc.conf | 2 +- > .../{de0-nano-soc.wks => de0-nano-soc.wks.in} | 2 +- > meta/classes/wic-img.bbclass | 53 +++++++++++++++++++++- > 4 files changed, 55 insertions(+), 4 deletions(-) > rename meta-isar/scripts/lib/wic/canned-wks/{de0-nano-soc.wks => de0-nano-soc.wks.in} (63%) > > diff --git a/doc/user_manual.md b/doc/user_manual.md > index ebc31c6..252d2ef 100644 > --- a/doc/user_manual.md > +++ b/doc/user_manual.md > @@ -197,6 +197,8 @@ A bootable disk image is generated if you set IMAGE_TYPE to 'wic-img'. Behind th > $ bitbake multiconfig:qemuamd64-stretch:isar-image-base > ``` > > +Variables may be used in `.wks.in` files; Isar will expand them and generate a regular `.wks` file before generating the disk image using `wic`. > + > In order to run the EFI images with `qemu`, an EFI firmware is required and available at the following address: > https://github.com/tianocore/edk2/tree/3858b4a1ff09d3243fea8d07bd135478237cb8f7 > > diff --git a/meta-isar/conf/machine/de0-nano-soc.conf b/meta-isar/conf/machine/de0-nano-soc.conf > index 63b2cfb..3a2c009 100644 > --- a/meta-isar/conf/machine/de0-nano-soc.conf > +++ b/meta-isar/conf/machine/de0-nano-soc.conf > @@ -11,7 +11,7 @@ U_BOOT_CONFIG_de0-nano-soc = "socfpga_de0_nano_soc_defconfig" > U_BOOT_BIN_de0-nano-soc = "u-boot-with-spl.sfp" > > IMAGE_TYPE ?= "wic-img" > -WKS_FILE ?= "de0-nano-soc" > +WKS_FILE ?= "de0-nano-soc.wks.in" > IMAGER_INSTALL += "u-boot-de0-nano-soc" > IMAGER_BUILD_DEPS += "u-boot-de0-nano-soc" > > diff --git a/meta-isar/scripts/lib/wic/canned-wks/de0-nano-soc.wks b/meta-isar/scripts/lib/wic/canned-wks/de0-nano-soc.wks.in > similarity index 63% > rename from meta-isar/scripts/lib/wic/canned-wks/de0-nano-soc.wks > rename to meta-isar/scripts/lib/wic/canned-wks/de0-nano-soc.wks.in > index 184474d..05ce61b 100644 > --- a/meta-isar/scripts/lib/wic/canned-wks/de0-nano-soc.wks > +++ b/meta-isar/scripts/lib/wic/canned-wks/de0-nano-soc.wks.in > @@ -3,7 +3,7 @@ > # > # SPDX-License-Identifier: MIT > > -part --source rawcopy --sourceparams "file=/usr/lib/u-boot/de0-nano-soc/u-boot-with-spl.sfp" --system-id 0xa2 --align 1 > +part --source rawcopy --sourceparams "file=/usr/lib/u-boot/${MACHINE}/${U_BOOT_BIN}" --system-id 0xa2 --align 1 > > part / --source rootfs-u-boot --ondisk mmcblk0 --fstype ext4 --label platform --align 1024 --active > > diff --git a/meta/classes/wic-img.bbclass b/meta/classes/wic-img.bbclass > index 76602d8..c9d90a9 100644 > --- a/meta/classes/wic-img.bbclass > +++ b/meta/classes/wic-img.bbclass > @@ -4,13 +4,36 @@ > # this class is heavily inspired by OEs ./meta/classes/image_types_wic.bbclass > # > > +WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' % os.path.exists('${WKS_FULL_PATH}')}" > + > +python do_write_wks_template () { > + """Write out expanded template contents to WKS_FULL_PATH.""" > + import re > + > + template_body = d.getVar('_WKS_TEMPLATE') > + > + # Remove any remnant variable references left behind by the expansion > + # due to undefined variables > + expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}") > + while True: > + new_body = re.sub(expand_var_regexp, '', template_body) > + if new_body == template_body: > + break > + else: > + template_body = new_body > + > + wks_file = d.getVar('WKS_FULL_PATH') > + with open(wks_file, 'w') as f: > + f.write(template_body) > +} > + > python () { > wks_full_path = None > > wks_file = d.getVar('WKS_FILE', True) > if not wks_file: > bb.fatal("WKS_FILE must be set") > - if not wks_file.endswith('.wks'): > + if not wks_file.endswith('.wks') and not wks_file.endswith('.wks.in'): > wks_file += '.wks' > > if os.path.isabs(wks_file): > @@ -28,6 +51,32 @@ python () { > bb.fatal("WKS_FILE '%s' not found" % wks_file) > > d.setVar('WKS_FULL_PATH', wks_full_path) > + > + wks_file_u = wks_full_path > + wks_file = wks_full_path > + base, ext = os.path.splitext(wks_file) > + if ext == '.in' and os.path.exists(wks_file): > + wks_out_file = os.path.join(d.getVar('WORKDIR'), os.path.basename(base)) > + d.setVar('WKS_FULL_PATH', wks_out_file) > + d.setVar('WKS_TEMPLATE_PATH', wks_file_u) > + d.setVar('WKS_FILE_CHECKSUM', '${WKS_TEMPLATE_PATH}:True') > + > + # We need to re-parse each time the file changes, and bitbake > + # needs to be told about that explicitly. > + bb.parse.mark_dependency(d, wks_file) > + > + try: > + with open(wks_file, 'r') as f: > + body = f.read() > + except (IOError, OSError) as exc: > + pass > + else: > + # Previously, I used expandWithRefs to get the dependency list > + # and add it to WICVARS, but there's no point re-parsing the > + # file in process_wks_template as well, so just put it in > + # a variable and let the metadata deal with the deps. > + d.setVar('_WKS_TEMPLATE', body) > + bb.build.addtask('do_write_wks_template', 'do_wic_image', None, d) > } > > inherit buildchroot > @@ -108,7 +157,7 @@ do_wic_image() { > cp -f $(ls -t -1 ${BUILDCHROOT_DIR}/tmp/${IMAGE_FULLNAME}.wic/*.direct | head -1) ${WIC_IMAGE_FILE} > } > > -do_wic_image[file-checksums] += "${WKS_FULL_PATH}:True" > +do_wic_image[file-checksums] += "${WKS_FILE_CHECKSUM}" > do_wic_image[depends] = "buildchroot-target:do_build" > > addtask wic_image before do_build after do_install_imager_deps > -- Maxim Osipov ilbers GmbH Maria-Merian-Str. 8 85521 Ottobrunn Germany +49 (151) 6517 6917 mosipov@ilbers.de http://ilbers.de/ Commercial register Munich, HRB 214197 General Manager: Baurzhan Ismagulov