From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6491554293275951104 X-Received: by 10.223.190.11 with SMTP id n11mr2072245wrh.2.1511770543601; Mon, 27 Nov 2017 00:15:43 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 10.223.186.10 with SMTP id o10ls3217579wrg.13.gmail; Mon, 27 Nov 2017 00:15:43 -0800 (PST) X-Google-Smtp-Source: AGs4zMauZoeuEcAnM3WptyXVwX6CAy4xblouS41OGg6OjMXM3M9sqbexS3xnI+SvzkO8GzC/aGap X-Received: by 10.223.157.206 with SMTP id q14mr2037556wre.11.1511770543284; Mon, 27 Nov 2017 00:15:43 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1511770543; cv=none; d=google.com; s=arc-20160816; b=RPtP4QQClpTDBGwzl4XqJGQHJ149W8lJRH6kdtQbIejUfbP9lK5898tqemSvqeG9+0 hoTsyH3z7VGhlr9tu4FijyVmy0Vr2hJsyI4E1cNYDU4NwomKOPNV9PDIP27TWRd+B6ga 0KosxwDG0KdzXANLckXC4s2rXqDkyk9plh/AvoIKhvgv25XYHX86oJkxrA65EmEHApJp 2Nj0q/R9taemDS0QqTMCRG6Qf8l9ZW8FLBD5qTR2R6f+59uuwx/v5k25EFi8hIfG6GFX cfTVtRVyYIo87lzCApzBN/sMcg0V6s3zegG6KAtI9yHuVIQ8rT+ZCDvfX0QhoYjndXQL IApg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject :arc-authentication-results; bh=USM0D7TIdZegdpeh8cTrmYnetKxgQT48Br1e3x6Lrq8=; b=DbnGGGYo90L6p0v4lzG6+41NTx7JxrVdBacci4Kfg5rabM40S7snFTxcdHpz4iEy8f l1haPdFep30MiCHTPIlP6VH58mnrABCtD3A3mlpFH4lYqSFCpRFmIut1wZ89VHgQb1WO 5YJ6yE1LYd+VDVYoDtqL0BjS3R46Bl+KIJ4G6969LqdjATfNDd6VbYGZLoKMPA43OhP1 4E/Z6DCZYhd0ZlL46EhoQbrspj1Wgi6CzU2r+T08HxYl4aghebOGwRMOUary/S+b3TGj AtFJyMLFyn+J74MuUsdtInAO2AHoibitWPpT9ctAjvTaP8ya+7BVwTP1YnV+7HDcpNO3 9EAw== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Return-Path: Received: from aqmola.ilbers.de (aqmola.ilbers.de. [85.214.62.211]) by gmr-mx.google.com with ESMTPS id t37si90133wrc.3.2017.11.27.00.15.43 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 27 Nov 2017 00:15:43 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) client-ip=85.214.62.211; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: best guess record for domain of asmirnov@ilbers.de designates 85.214.62.211 as permitted sender) smtp.mailfrom=asmirnov@ilbers.de Received: from [10.0.2.15] ([188.227.110.165]) (authenticated bits=0) by aqmola.ilbers.de (8.14.4/8.14.4/Debian-4+deb7u1) with ESMTP id vAR8FeT4018262 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES128-SHA bits=128 verify=NOT); Mon, 27 Nov 2017 09:15:42 +0100 Subject: Re: [PATCH 3/4 v2] buildchroot: Add prepare and cleanup tasks To: Henning Schild Cc: isar-users@googlegroups.com References: <20171123102532.26274-1-asmirnov@ilbers.de> <20171123102532.26274-4-asmirnov@ilbers.de> <20171124173408.7a2cc881@md1em3qc> <1456b2b1-7c28-6fa6-11a9-74d851417af2@ilbers.de> <20171127085619.755c58f5@md1em3qc> From: Alexander Smirnov Message-ID: Date: Mon, 27 Nov 2017 11:15:34 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.3.0 MIME-Version: 1.0 In-Reply-To: <20171127085619.755c58f5@md1em3qc> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: U3LRDp1XF8JI On 11/27/2017 10:56 AM, Henning Schild wrote: > Am Fri, 24 Nov 2017 19:51:26 +0300 > schrieb Alexander Smirnov : > >> On 11/24/2017 07:34 PM, Henning Schild wrote: >>> Am Thu, 23 Nov 2017 13:25:31 +0300 >>> schrieb Alexander Smirnov : >>> >>>> Some packages could require builchroot filesystem tunning, for >>>> example share Isar downloads folder with chroot filesystem. >>>> >>>> This patch adds two tasks for buildchroot: >>>> >>>> 1. 'do_prepare': This task is executed after buildchroot rootfs >>>> generation and before 'do_build' from packages to be built. >>>> Some notes: >>>> - An individual task is required for the case when you want to >>>> rebuild some package, this would require to run this preparation >>>> stuff for existing buildchroot rootfs. >>>> - This task should not have stamp. This means that anytime you >>>> want to build something, 'do_prepare' should be executed. >>>> >>>> 2. 'do_cleanup': This task is executed after all the packages are >>>> deployed. Some notes: >>>> - This task also should not have stamp. >>>> - This task depends from the recipes listed in IMAGE_INSTALL. >>>> >>>> Signed-off-by: Alexander Smirnov >>>> --- >>>> meta/classes/dpkg-base.bbclass | 2 +- >>>> meta/classes/image.bbclass | 1 + >>>> meta/recipes-devtools/buildchroot/buildchroot.bb | 21 >>>> +++++++++++++++++++++ 3 files changed, 23 insertions(+), 1 >>>> deletion(-) >>>> >>>> diff --git a/meta/classes/dpkg-base.bbclass >>>> b/meta/classes/dpkg-base.bbclass index 4941f9b..21ae88d 100644 >>>> --- a/meta/classes/dpkg-base.bbclass >>>> +++ b/meta/classes/dpkg-base.bbclass >>>> @@ -2,7 +2,7 @@ >>>> # Copyright (C) 2017 Siemens AG >>>> >>>> # Add dependency from buildchroot creation >>>> -do_build[depends] = "buildchroot:do_build" >>>> +do_build[depends] = "buildchroot:do_prepare" >>>> >>>> # Each package should have its own unique build folder, so use >>>> # recipe name as identifier >>>> diff --git a/meta/classes/image.bbclass >>>> b/meta/classes/image.bbclass index 7813b16..b0f645c 100644 >>>> --- a/meta/classes/image.bbclass >>>> +++ b/meta/classes/image.bbclass >>>> @@ -83,6 +83,7 @@ do_populate() { >>>> >>>> addtask populate before do_build after do_unpack >>>> do_populate[deptask] = "do_deploy_deb" >>>> +do_populate[depends] = "buildchroot:do_cleanup" >>> >>> Why do we need that? To me that looks redundant. >> >> 1. buildchroot:do_cleanup task should be executed after all the >> packages are build, so that's why the following is used: >> >> DEPENDS += "${IMAGE_INSTALL}" >> do_cleanup[deptask] = "do_deploy_deb" > > That one is clear. > >> 2. buildchroot:do_cleanup should be somehow integrated to Isar >> pipeline, so some dependency from late task (which runs after >> packages building) should be added. I chose image:do_populate, so the >> following line forces Isar to run do_cleanup during build: >> >> do_populate[depends] = "buildchroot:do_cleanup" > > If you do not depend on it in the image-recipe, will it not be executed? It will not, buildchroot is joint to pipeline only by 'do_build' task, so all the tasks after are silently ignored. For sure, I've tested this. > Please remember that we discussed dropping do_populate, so it might be > a good idea to integrate that step somewhere else. Maybe give the image > its own do_cleanup. > > https://groups.google.com/d/msg/isar-users/efer3RF989o/mB5Fce9WBgAJ > I remember this discussion, but at the moment I have no clear answer whether do_populate will go or not. The concept that was discussed contains the following steps: 1. Buildchroot generation. 2. Building all the packages and put them to apt. 3. Provide this apt as the input for image's multistrap. Thinking on this I've realized that this approach has one issue - it serializes buildchroot and image generation, so significantly increases Isar build time. Possible solution could be: 1. Buildchroot and base image rootfs generation. 2. Building all the packages and put them to apt. 3. Populate image rootfs via apt-get using Isar apt. I'd like to discuss this in more details when I return to apt topic. So for now I'd rely on existing pipeline. Alex > Henning > >> Alex >> >>> >>> Henning >>> >>>> do_rootfs[stamp-extra-info] = "${MACHINE}-${DISTRO}" >>>> >>>> diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb >>>> b/meta/recipes-devtools/buildchroot/buildchroot.bb index >>>> 6a94733..fe88e98 100644 --- >>>> a/meta/recipes-devtools/buildchroot/buildchroot.bb +++ >>>> b/meta/recipes-devtools/buildchroot/buildchroot.bb @@ -67,7 +67,28 >>>> @@ do_build() { # Install package builder script >>>> sudo install -m 755 ${WORKDIR}/build.sh ${BUILDCHROOT_DIR} >>>> >>>> + # Create share point for downloads >>>> + sudo install -d ${BUILDCHROOT_DIR}/git >>>> + >>>> # Configure root filesystem >>>> sudo chroot ${BUILDCHROOT_DIR} /configscript.sh >>>> _do_build_cleanup >>>> } >>>> + >>>> +do_prepare[nostamp] = "1" >>>> + >>>> +do_prepare() { >>>> + sudo mount --bind ${GITDIR} ${BUILDCHROOT_DIR}/git >>>> +} >>>> + >>>> +addtask prepare after do_build >>>> + >>>> +DEPENDS += "${IMAGE_INSTALL}" >>>> +do_cleanup[deptask] = "do_deploy_deb" >>>> +do_cleanup[nostamp] = "1" >>>> + >>>> +do_cleanup() { >>>> + sudo umount ${BUILDCHROOT_DIR}/git >>>> +} >>>> + >>>> +addtask cleanup after do_prepare