From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6655521619015892992 X-Received: by 2002:a17:906:2346:: with SMTP id m6mr2863911eja.15.1549653235106; Fri, 08 Feb 2019 11:13:55 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a17:906:d1d5:: with SMTP id bs21ls1343256ejb.9.gmail; Fri, 08 Feb 2019 11:13:54 -0800 (PST) X-Google-Smtp-Source: AHgI3IZOiaghQ3kpeu30EmCOiCwKptpKms+Qg+ctItw5moIomAAOCE6cxLoHznwFM5z1mQ+0mtBQ X-Received: by 2002:a17:906:2641:: with SMTP id i1mr1924226ejc.9.1549653234668; Fri, 08 Feb 2019 11:13:54 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1549653234; cv=none; d=google.com; s=arc-20160816; b=rLwIKds8AKVT5laydG7tKjfBr1OEOVyiAWOR/jPM0MMcQ+2/OltfH/Nf0pvIV8EZJk rUoKsTZcJYOPXL4Cf0QKQ/SabRbkM2XaPG2C2m5CGPyYa5fdAO1gf8slOoDhBoL4s+Xx 9HbJvnpiTuQIcex9s80tTc+HkVWwLS8JLEyycGDq80FPJZpkp8hCvFaH3jNf690BnJo6 k4HEuMBoRwm8HpUOinM9Aq4noL5OvoQWgLTd8kVQyv8mV42QwZyzLfFVUXepHV5njY3X PjUcMxIZKp6TxLobIsSjrKEFBbz/1Kp5+qigHAAjIyQmW9I6Nio3dUWqIka/yO/UQQRh dFTg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=mime-version:references:in-reply-to:message-id:date:subject:cc:to :from; bh=XuHwgUYUWmZVMe1TsLDh7gL7cEC+uvEjDgpkJLuV+1Q=; b=n/7PmG+VlHKvW11eidc6uU5YZhlPkHkxk7QtG6X4OYrkRZL0RZ1l0quUe5aU9D3Lj+ dKFl+9wRLOW1qjA0dtaiOxC2e6HsFlqPs8/HQ48VdtAk1n198FrzDcGtUykZQAU8k3Ci VxAMXUf0avhe/mweaseoyizkAtvTv7UqbbwcyaFCA2nebEQFEiW0fsmMjBPuvDiTqWYS JT8+2lIDAbFDqBxw30yQfkkh4LAsN4K/f2MbZSjFBHQy+zP9SZYFZ9ED5SwEt51t/QSv /R5PgxkE+8DYsyMA/Wu26IjptltTViNP+FO3TrulrP6qnLjG9XIgniPSOhnRYCdGzXh2 C4+w== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of cedric_hombourger@mentor.com designates 192.94.38.131 as permitted sender) smtp.mailfrom=Cedric_Hombourger@mentor.com Return-Path: Received: from relay1.mentorg.com (relay1.mentorg.com. [192.94.38.131]) by gmr-mx.google.com with ESMTPS id i17si114625edg.2.2019.02.08.11.13.54 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Fri, 08 Feb 2019 11:13:54 -0800 (PST) Received-SPF: pass (google.com: domain of cedric_hombourger@mentor.com designates 192.94.38.131 as permitted sender) client-ip=192.94.38.131; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of cedric_hombourger@mentor.com designates 192.94.38.131 as permitted sender) smtp.mailfrom=Cedric_Hombourger@mentor.com Received: from nat-ies.mentorg.com ([192.94.31.2] helo=svr-ies-mbx-02.mgc.mentorg.com) by relay1.mentorg.com with esmtps (TLSv1.2:ECDHE-RSA-AES256-SHA384:256) id 1gsBaz-00015C-2s from Cedric_Hombourger@mentor.com for isar-users@googlegroups.com; Fri, 08 Feb 2019 11:13:53 -0800 Received: from FRG-W10-HOMBOUR.world.mentorg.com (137.202.0.90) by svr-ies-mbx-02.mgc.mentorg.com (139.181.222.2) with Microsoft SMTP Server (TLS) id 15.0.1320.4; Fri, 8 Feb 2019 19:13:44 +0000 From: Cedric Hombourger To: CC: Cedric Hombourger Subject: [PATCH v2] wic-img: import support for .wks.in files from oe-core Date: Fri, 8 Feb 2019 20:13:06 +0100 Message-ID: <1549653186-1216-1-git-send-email-Cedric_Hombourger@mentor.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: References: MIME-Version: 1.0 Content-Type: text/plain X-Originating-IP: [137.202.0.90] X-ClientProxiedBy: SVR-IES-MBX-03.mgc.mentorg.com (139.181.222.3) To svr-ies-mbx-02.mgc.mentorg.com (139.181.222.2) X-TUID: Ps5yz4jNOQ7f OpenEmbedded supports use of variables in wks files if they are suffixed with .wks.in (instead of .wks). Use of this suffix adds a task to the build process to expand variables to their values when known or to the empty string when they are not. The result of the evaluation is placed in a regular .wks file placed in the build folder. The .wks file for the de0-nano-soc machine was modified to show use of this feature and get it regularly tested as part of our regular Isar builds. Signed-off-by: Cedric Hombourger --- doc/user_manual.md | 2 + meta-isar/conf/machine/de0-nano-soc.conf | 2 +- .../{de0-nano-soc.wks => de0-nano-soc.wks.in} | 2 +- meta/classes/wic-img.bbclass | 53 +++++++++++++++++++++- 4 files changed, 55 insertions(+), 4 deletions(-) rename meta-isar/scripts/lib/wic/canned-wks/{de0-nano-soc.wks => de0-nano-soc.wks.in} (63%) diff --git a/doc/user_manual.md b/doc/user_manual.md index ebc31c6..252d2ef 100644 --- a/doc/user_manual.md +++ b/doc/user_manual.md @@ -197,6 +197,8 @@ A bootable disk image is generated if you set IMAGE_TYPE to 'wic-img'. Behind th $ bitbake multiconfig:qemuamd64-stretch:isar-image-base ``` +Variables may be used in `.wks.in` files; Isar will expand them and generate a regular `.wks` file before generating the disk image using `wic`. + In order to run the EFI images with `qemu`, an EFI firmware is required and available at the following address: https://github.com/tianocore/edk2/tree/3858b4a1ff09d3243fea8d07bd135478237cb8f7 diff --git a/meta-isar/conf/machine/de0-nano-soc.conf b/meta-isar/conf/machine/de0-nano-soc.conf index 63b2cfb..3a2c009 100644 --- a/meta-isar/conf/machine/de0-nano-soc.conf +++ b/meta-isar/conf/machine/de0-nano-soc.conf @@ -11,7 +11,7 @@ U_BOOT_CONFIG_de0-nano-soc = "socfpga_de0_nano_soc_defconfig" U_BOOT_BIN_de0-nano-soc = "u-boot-with-spl.sfp" IMAGE_TYPE ?= "wic-img" -WKS_FILE ?= "de0-nano-soc" +WKS_FILE ?= "de0-nano-soc.wks.in" IMAGER_INSTALL += "u-boot-de0-nano-soc" IMAGER_BUILD_DEPS += "u-boot-de0-nano-soc" diff --git a/meta-isar/scripts/lib/wic/canned-wks/de0-nano-soc.wks b/meta-isar/scripts/lib/wic/canned-wks/de0-nano-soc.wks.in similarity index 63% rename from meta-isar/scripts/lib/wic/canned-wks/de0-nano-soc.wks rename to meta-isar/scripts/lib/wic/canned-wks/de0-nano-soc.wks.in index 184474d..05ce61b 100644 --- a/meta-isar/scripts/lib/wic/canned-wks/de0-nano-soc.wks +++ b/meta-isar/scripts/lib/wic/canned-wks/de0-nano-soc.wks.in @@ -3,7 +3,7 @@ # # SPDX-License-Identifier: MIT -part --source rawcopy --sourceparams "file=/usr/lib/u-boot/de0-nano-soc/u-boot-with-spl.sfp" --system-id 0xa2 --align 1 +part --source rawcopy --sourceparams "file=/usr/lib/u-boot/${MACHINE}/${U_BOOT_BIN}" --system-id 0xa2 --align 1 part / --source rootfs-u-boot --ondisk mmcblk0 --fstype ext4 --label platform --align 1024 --active diff --git a/meta/classes/wic-img.bbclass b/meta/classes/wic-img.bbclass index 76602d8..c9d90a9 100644 --- a/meta/classes/wic-img.bbclass +++ b/meta/classes/wic-img.bbclass @@ -4,13 +4,36 @@ # this class is heavily inspired by OEs ./meta/classes/image_types_wic.bbclass # +WKS_FILE_CHECKSUM = "${@'${WKS_FULL_PATH}:%s' % os.path.exists('${WKS_FULL_PATH}')}" + +python do_write_wks_template () { + """Write out expanded template contents to WKS_FULL_PATH.""" + import re + + template_body = d.getVar('_WKS_TEMPLATE') + + # Remove any remnant variable references left behind by the expansion + # due to undefined variables + expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}") + while True: + new_body = re.sub(expand_var_regexp, '', template_body) + if new_body == template_body: + break + else: + template_body = new_body + + wks_file = d.getVar('WKS_FULL_PATH') + with open(wks_file, 'w') as f: + f.write(template_body) +} + python () { wks_full_path = None wks_file = d.getVar('WKS_FILE', True) if not wks_file: bb.fatal("WKS_FILE must be set") - if not wks_file.endswith('.wks'): + if not wks_file.endswith('.wks') and not wks_file.endswith('.wks.in'): wks_file += '.wks' if os.path.isabs(wks_file): @@ -28,6 +51,32 @@ python () { bb.fatal("WKS_FILE '%s' not found" % wks_file) d.setVar('WKS_FULL_PATH', wks_full_path) + + wks_file_u = wks_full_path + wks_file = wks_full_path + base, ext = os.path.splitext(wks_file) + if ext == '.in' and os.path.exists(wks_file): + wks_out_file = os.path.join(d.getVar('WORKDIR'), os.path.basename(base)) + d.setVar('WKS_FULL_PATH', wks_out_file) + d.setVar('WKS_TEMPLATE_PATH', wks_file_u) + d.setVar('WKS_FILE_CHECKSUM', '${WKS_TEMPLATE_PATH}:True') + + # We need to re-parse each time the file changes, and bitbake + # needs to be told about that explicitly. + bb.parse.mark_dependency(d, wks_file) + + try: + with open(wks_file, 'r') as f: + body = f.read() + except (IOError, OSError) as exc: + pass + else: + # Previously, I used expandWithRefs to get the dependency list + # and add it to WICVARS, but there's no point re-parsing the + # file in process_wks_template as well, so just put it in + # a variable and let the metadata deal with the deps. + d.setVar('_WKS_TEMPLATE', body) + bb.build.addtask('do_write_wks_template', 'do_wic_image', None, d) } inherit buildchroot @@ -108,7 +157,7 @@ do_wic_image() { cp -f $(ls -t -1 ${BUILDCHROOT_DIR}/tmp/${IMAGE_FULLNAME}.wic/*.direct | head -1) ${WIC_IMAGE_FILE} } -do_wic_image[file-checksums] += "${WKS_FULL_PATH}:True" +do_wic_image[file-checksums] += "${WKS_FILE_CHECKSUM}" do_wic_image[depends] = "buildchroot-target:do_build" addtask wic_image before do_build after do_install_imager_deps -- 2.11.0