* [PATCH 1/6] Fix indention of base_do_build
2018-02-12 7:59 [PATCH 0/6] Add support for automatic partial rebuilds on recipe changes Jan Kiszka
@ 2018-02-12 7:59 ` Jan Kiszka
2018-02-12 7:59 ` [PATCH 2/6] Add clean and cleanall tasks Jan Kiszka
` (6 subsequent siblings)
7 siblings, 0 replies; 27+ messages in thread
From: Jan Kiszka @ 2018-02-12 7:59 UTC (permalink / raw)
To: isar-users
From: Jan Kiszka <jan.kiszka@siemens.com>
Python code must not contain tabs.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
meta/classes/base.bbclass | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index ae84677..0fa27db 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -106,8 +106,8 @@ addtask unpack after do_fetch before do_build
addtask build
do_build[dirs] = "${TOPDIR}"
python base_do_build () {
- bb.note("The included, default BB base.bbclass does not define a useful default task.")
- bb.note("Try running the 'listtasks' task against a .bb to see what tasks are defined.")
+ bb.note("The included, default BB base.bbclass does not define a useful default task.")
+ bb.note("Try running the 'listtasks' task against a .bb to see what tasks are defined.")
}
EXPORT_FUNCTIONS do_clean do_mrproper do_build
--
2.13.6
^ permalink raw reply [flat|nested] 27+ messages in thread* [PATCH 2/6] Add clean and cleanall tasks
2018-02-12 7:59 [PATCH 0/6] Add support for automatic partial rebuilds on recipe changes Jan Kiszka
2018-02-12 7:59 ` [PATCH 1/6] Fix indention of base_do_build Jan Kiszka
@ 2018-02-12 7:59 ` Jan Kiszka
2018-02-13 12:49 ` Alexander Smirnov
2018-02-12 7:59 ` [PATCH 3/6] Enable recipe caching Jan Kiszka
` (5 subsequent siblings)
7 siblings, 1 reply; 27+ messages in thread
From: Jan Kiszka @ 2018-02-12 7:59 UTC (permalink / raw)
To: isar-users
From: Jan Kiszka <jan.kiszka@siemens.com>
Derived from OE, this adds clean and cleanall tasks for Isar. The clean
task comes with CLEANFUNCS hook, we just run them prior to cleaning the
WORKDIR. This is used by dpkg-base to remove all packages from isar-apt
that the build may have deployed.
As we are at it, also do a repo clean prior to deploying a package -
just in case something triggers a partial rebuild without prior
cleaning...
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
meta/classes/base.bbclass | 35 ++++++++++++++++++++++++++++++++++-
meta/classes/dpkg-base.bbclass | 14 ++++++++++++++
2 files changed, 48 insertions(+), 1 deletion(-)
diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
index 0fa27db..b0e174e 100644
--- a/meta/classes/base.bbclass
+++ b/meta/classes/base.bbclass
@@ -110,4 +110,37 @@ python base_do_build () {
bb.note("Try running the 'listtasks' task against a .bb to see what tasks are defined.")
}
-EXPORT_FUNCTIONS do_clean do_mrproper do_build
+EXPORT_FUNCTIONS do_build
+
+CLEANFUNCS ?= ""
+
+# Derived from OpenEmbedded Core: meta/classes/utility-tasks.bbclass
+addtask clean
+do_clean[nostamp] = "1"
+python do_clean() {
+ import subprocess
+
+ for f in (d.getVar('CLEANFUNCS', True) or '').split():
+ bb.build.exec_func(f, d)
+
+ dir = d.expand("${WORKDIR}")
+ subprocess.call('sudo rm -rf ' + dir, shell=True)
+
+ dir = "%s.*" % bb.data.expand(d.getVar('STAMP', False), d)
+ subprocess.call('sudo rm -rf ' + dir, shell=True)
+}
+
+# Derived from OpenEmbedded Core: meta/classes/base.bbclass
+addtask cleanall after do_clean
+do_cleanall[nostamp] = "1"
+python do_cleanall() {
+ src_uri = (d.getVar('SRC_URI', True) or "").split()
+ if len(src_uri) == 0:
+ return
+
+ try:
+ fetcher = bb.fetch2.Fetch(src_uri, d)
+ fetcher.clean()
+ except bb.fetch2.BBFetchException as e:
+ bb.fatal(str(e))
+}
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index 69c8ad5..a45fbce 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -31,8 +31,22 @@ do_build() {
sudo rmdir ${BUILDROOT} 2>/dev/null || true
}
+CLEANFUNCS += "repo_clean"
+
+repo_clean() {
+ PACKAGES=$(cd ${WORKDIR}; ls *.deb | sed 's/\([^_]*\).*/\1/')
+ if [ -n "${PACKAGES}" ]; then
+ reprepro -b ${DEPLOY_DIR_APT}/${DISTRO} \
+ --dbdir ${DEPLOY_DIR_DB}/${DISTRO} \
+ -C main \
+ remove ${DEBDISTRONAME} \
+ ${PACKAGES}
+ fi
+}
+
# Install package to Isar-apt
do_deploy_deb() {
+ repo_clean
reprepro -b ${DEPLOY_DIR_APT}/${DISTRO} \
--dbdir ${DEPLOY_DIR_DB}/${DISTRO} \
-C main \
--
2.13.6
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 2/6] Add clean and cleanall tasks
2018-02-12 7:59 ` [PATCH 2/6] Add clean and cleanall tasks Jan Kiszka
@ 2018-02-13 12:49 ` Alexander Smirnov
2018-02-13 12:51 ` Jan Kiszka
0 siblings, 1 reply; 27+ messages in thread
From: Alexander Smirnov @ 2018-02-13 12:49 UTC (permalink / raw)
To: Jan Kiszka, isar-users
On 02/12/2018 10:59 AM, Jan Kiszka wrote:
> From: Jan Kiszka <jan.kiszka@siemens.com>
>
> Derived from OE, this adds clean and cleanall tasks for Isar. The clean
> task comes with CLEANFUNCS hook, we just run them prior to cleaning the
> WORKDIR. This is used by dpkg-base to remove all packages from isar-apt
> that the build may have deployed.
>
> As we are at it, also do a repo clean prior to deploying a package -
> just in case something triggers a partial rebuild without prior
> cleaning...
>
Very good! :-)
I think, we also should erase the package's "PROVIDES" from buildchroot
(like libhello-dev). This could be implemented later in next series.
Alex
> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
> meta/classes/base.bbclass | 35 ++++++++++++++++++++++++++++++++++-
> meta/classes/dpkg-base.bbclass | 14 ++++++++++++++
> 2 files changed, 48 insertions(+), 1 deletion(-)
>
> diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass
> index 0fa27db..b0e174e 100644
> --- a/meta/classes/base.bbclass
> +++ b/meta/classes/base.bbclass
> @@ -110,4 +110,37 @@ python base_do_build () {
> bb.note("Try running the 'listtasks' task against a .bb to see what tasks are defined.")
> }
>
> -EXPORT_FUNCTIONS do_clean do_mrproper do_build
> +EXPORT_FUNCTIONS do_build
> +
> +CLEANFUNCS ?= ""
> +
> +# Derived from OpenEmbedded Core: meta/classes/utility-tasks.bbclass
> +addtask clean
> +do_clean[nostamp] = "1"
> +python do_clean() {
> + import subprocess
> +
> + for f in (d.getVar('CLEANFUNCS', True) or '').split():
> + bb.build.exec_func(f, d)
> +
> + dir = d.expand("${WORKDIR}")
> + subprocess.call('sudo rm -rf ' + dir, shell=True)
> +
> + dir = "%s.*" % bb.data.expand(d.getVar('STAMP', False), d)
> + subprocess.call('sudo rm -rf ' + dir, shell=True)
> +}
> +
> +# Derived from OpenEmbedded Core: meta/classes/base.bbclass
> +addtask cleanall after do_clean
> +do_cleanall[nostamp] = "1"
> +python do_cleanall() {
> + src_uri = (d.getVar('SRC_URI', True) or "").split()
> + if len(src_uri) == 0:
> + return
> +
> + try:
> + fetcher = bb.fetch2.Fetch(src_uri, d)
> + fetcher.clean()
> + except bb.fetch2.BBFetchException as e:
> + bb.fatal(str(e))
> +}
> diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
> index 69c8ad5..a45fbce 100644
> --- a/meta/classes/dpkg-base.bbclass
> +++ b/meta/classes/dpkg-base.bbclass
> @@ -31,8 +31,22 @@ do_build() {
> sudo rmdir ${BUILDROOT} 2>/dev/null || true
> }
>
> +CLEANFUNCS += "repo_clean"
> +
> +repo_clean() {
> + PACKAGES=$(cd ${WORKDIR}; ls *.deb | sed 's/\([^_]*\).*/\1/')
> + if [ -n "${PACKAGES}" ]; then
> + reprepro -b ${DEPLOY_DIR_APT}/${DISTRO} \
> + --dbdir ${DEPLOY_DIR_DB}/${DISTRO} \
> + -C main \
> + remove ${DEBDISTRONAME} \
> + ${PACKAGES}
> + fi
> +}
> +
> # Install package to Isar-apt
> do_deploy_deb() {
> + repo_clean
> reprepro -b ${DEPLOY_DIR_APT}/${DISTRO} \
> --dbdir ${DEPLOY_DIR_DB}/${DISTRO} \
> -C main \
>
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 2/6] Add clean and cleanall tasks
2018-02-13 12:49 ` Alexander Smirnov
@ 2018-02-13 12:51 ` Jan Kiszka
0 siblings, 0 replies; 27+ messages in thread
From: Jan Kiszka @ 2018-02-13 12:51 UTC (permalink / raw)
To: Alexander Smirnov, isar-users
On 2018-02-13 13:49, Alexander Smirnov wrote:
> On 02/12/2018 10:59 AM, Jan Kiszka wrote:
>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>
>> Derived from OE, this adds clean and cleanall tasks for Isar. The clean
>> task comes with CLEANFUNCS hook, we just run them prior to cleaning the
>> WORKDIR. This is used by dpkg-base to remove all packages from isar-apt
>> that the build may have deployed.
>>
>> As we are at it, also do a repo clean prior to deploying a package -
>> just in case something triggers a partial rebuild without prior
>> cleaning...
>>
>
> Very good! :-)
>
> I think, we also should erase the package's "PROVIDES" from buildchroot
> (like libhello-dev). This could be implemented later in next series.
Good point. Will try to look into that later.
Jan
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH 3/6] Enable recipe caching
2018-02-12 7:59 [PATCH 0/6] Add support for automatic partial rebuilds on recipe changes Jan Kiszka
2018-02-12 7:59 ` [PATCH 1/6] Fix indention of base_do_build Jan Kiszka
2018-02-12 7:59 ` [PATCH 2/6] Add clean and cleanall tasks Jan Kiszka
@ 2018-02-12 7:59 ` Jan Kiszka
2018-02-12 7:59 ` [PATCH 4/6] Enable proper rebuilds on dependency changes Jan Kiszka
` (4 subsequent siblings)
7 siblings, 0 replies; 27+ messages in thread
From: Jan Kiszka @ 2018-02-12 7:59 UTC (permalink / raw)
To: isar-users
From: Jan Kiszka <jan.kiszka@siemens.com>
Import BB_HASHBASE_WHITELIST and BB_HASHCONFIG_WHITELIST from OE,
removing some items that are surely OE-specific, and set CACHE. That's
all what's needed to enable bitbake's recipe parsing cache.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
meta/conf/isar-bitbake.conf | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/meta/conf/isar-bitbake.conf b/meta/conf/isar-bitbake.conf
index f7f751c..660e0c0 100644
--- a/meta/conf/isar-bitbake.conf
+++ b/meta/conf/isar-bitbake.conf
@@ -22,6 +22,21 @@ WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PF}"
DL_DIR = "${TOPDIR}/downloads"
SSTATE_DIR ?= "${TMPDIR}/sstate-cache"
BUILDCHROOT_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/buildchroot/rootfs"
+CACHE = "${TMPDIR}/cache"
+
+BB_HASHBASE_WHITELIST ?= "TMPDIR FILE PATH PWD BB_TASKHASH BBPATH BBSERVER DL_DIR \
+ SSTATE_DIR THISDIR FILESEXTRAPATHS FILE_DIRNAME HOME LOGNAME SHELL TERM \
+ USER FILESPATH STAGING_DIR_HOST STAGING_DIR_TARGET COREBASE PRSERV_HOST \
+ PRSERV_DUMPDIR PRSERV_DUMPFILE PRSERV_LOCKDOWN PARALLEL_MAKE \
+ CCACHE_DIR EXTERNAL_TOOLCHAIN CCACHE CCACHE_NOHASHDIR LICENSE_PATH SDKPKGSUFFIX \
+ WORKDIR STAMPCLEAN PKGDATA_DIR BUILD_ARCH SSTATE_PKGARCH \
+ BB_WORKERCONTEXT BB_LIMITEDDEPS DEPLOY_DIR"
+BB_HASHCONFIG_WHITELIST ?= "${BB_HASHBASE_WHITELIST} DATE TIME SSH_AGENT_PID \
+ SSH_AUTH_SOCK PSEUDO_BUILD BB_ENV_EXTRAWHITE DISABLE_SANITY_CHECKS \
+ BB_NUMBER_THREADS BB_ORIGENV BB_INVALIDCONF BBINCLUDED \
+ GIT_PROXY_COMMAND ALL_PROXY all_proxy NO_PROXY no_proxy FTP_PROXY ftp_proxy \
+ HTTP_PROXY http_proxy HTTPS_PROXY https_proxy SOCKS5_USER SOCKS5_PASSWD \
+ BB_SETSCENE_ENFORCE BB_CMDLINE BB_SERVER_TIMEOUT"
# Setup our default hash policy
BB_SIGNATURE_HANDLER ?= "noop"
--
2.13.6
^ permalink raw reply [flat|nested] 27+ messages in thread* [PATCH 4/6] Enable proper rebuilds on dependency changes
2018-02-12 7:59 [PATCH 0/6] Add support for automatic partial rebuilds on recipe changes Jan Kiszka
` (2 preceding siblings ...)
2018-02-12 7:59 ` [PATCH 3/6] Enable recipe caching Jan Kiszka
@ 2018-02-12 7:59 ` Jan Kiszka
2018-02-13 13:03 ` Alexander Smirnov
2018-02-12 7:59 ` [PATCH 5/6] dpkg-raw: Clean DEBIAN dir prior to filling it Jan Kiszka
` (3 subsequent siblings)
7 siblings, 1 reply; 27+ messages in thread
From: Jan Kiszka @ 2018-02-12 7:59 UTC (permalink / raw)
To: isar-users
From: Jan Kiszka <jan.kiszka@siemens.com>
Install a basichash as signature handler and set the stamp policy to
full - and suddenly we get proper rebuilds of the image and all affected
packages when some recipe down the dependency chain changed!
We are using the legacy bitbake mechanism here as we do not have
setscene machinery like OE. Still good enough for us.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
meta/conf/isar-bitbake.conf | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/meta/conf/isar-bitbake.conf b/meta/conf/isar-bitbake.conf
index 660e0c0..c01e486 100644
--- a/meta/conf/isar-bitbake.conf
+++ b/meta/conf/isar-bitbake.conf
@@ -37,9 +37,8 @@ BB_HASHCONFIG_WHITELIST ?= "${BB_HASHBASE_WHITELIST} DATE TIME SSH_AGENT_PID \
GIT_PROXY_COMMAND ALL_PROXY all_proxy NO_PROXY no_proxy FTP_PROXY ftp_proxy \
HTTP_PROXY http_proxy HTTPS_PROXY https_proxy SOCKS5_USER SOCKS5_PASSWD \
BB_SETSCENE_ENFORCE BB_CMDLINE BB_SERVER_TIMEOUT"
-
-# Setup our default hash policy
-BB_SIGNATURE_HANDLER ?= "noop"
+BB_SIGNATURE_HANDLER ?= "basichash"
+BB_STAMP_POLICY ?= "full"
# Add event handlers for bitbake
INHERIT += "isar-events"
--
2.13.6
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 4/6] Enable proper rebuilds on dependency changes
2018-02-12 7:59 ` [PATCH 4/6] Enable proper rebuilds on dependency changes Jan Kiszka
@ 2018-02-13 13:03 ` Alexander Smirnov
2018-02-13 16:21 ` Jan Kiszka
0 siblings, 1 reply; 27+ messages in thread
From: Alexander Smirnov @ 2018-02-13 13:03 UTC (permalink / raw)
To: Jan Kiszka, isar-users
On 02/12/2018 10:59 AM, Jan Kiszka wrote:
> From: Jan Kiszka <jan.kiszka@siemens.com>
>
> Install a basichash as signature handler and set the stamp policy to
> full - and suddenly we get proper rebuilds of the image and all affected
> packages when some recipe down the dependency chain changed!
>
> We are using the legacy bitbake mechanism here as we do not have
> setscene machinery like OE. Still good enough for us.
>
BTW: have you tried this? Two years ago I tried it in Yocto and it
didn't work properly, not all the dependents were rebuilt. If it works,
it's a very good feature!
Also do you know if this feature relies on DEPENDS variable? If so, we
should keep this in mind with the idea to drop DEPENDS from recipes.
Alex
> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
> meta/conf/isar-bitbake.conf | 5 ++---
> 1 file changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/meta/conf/isar-bitbake.conf b/meta/conf/isar-bitbake.conf
> index 660e0c0..c01e486 100644
> --- a/meta/conf/isar-bitbake.conf
> +++ b/meta/conf/isar-bitbake.conf
> @@ -37,9 +37,8 @@ BB_HASHCONFIG_WHITELIST ?= "${BB_HASHBASE_WHITELIST} DATE TIME SSH_AGENT_PID \
> GIT_PROXY_COMMAND ALL_PROXY all_proxy NO_PROXY no_proxy FTP_PROXY ftp_proxy \
> HTTP_PROXY http_proxy HTTPS_PROXY https_proxy SOCKS5_USER SOCKS5_PASSWD \
> BB_SETSCENE_ENFORCE BB_CMDLINE BB_SERVER_TIMEOUT"
> -
> -# Setup our default hash policy
> -BB_SIGNATURE_HANDLER ?= "noop"
> +BB_SIGNATURE_HANDLER ?= "basichash"
> +BB_STAMP_POLICY ?= "full"
>
> # Add event handlers for bitbake
> INHERIT += "isar-events"
>
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 4/6] Enable proper rebuilds on dependency changes
2018-02-13 13:03 ` Alexander Smirnov
@ 2018-02-13 16:21 ` Jan Kiszka
2018-02-13 17:57 ` Jan Kiszka
0 siblings, 1 reply; 27+ messages in thread
From: Jan Kiszka @ 2018-02-13 16:21 UTC (permalink / raw)
To: Alexander Smirnov, isar-users
On 2018-02-13 14:03, Alexander Smirnov wrote:
> On 02/12/2018 10:59 AM, Jan Kiszka wrote:
>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>
>> Install a basichash as signature handler and set the stamp policy to
>> full - and suddenly we get proper rebuilds of the image and all affected
>> packages when some recipe down the dependency chain changed!
>>
>> We are using the legacy bitbake mechanism here as we do not have
>> setscene machinery like OE. Still good enough for us.
>>
>
> BTW: have you tried this? Two years ago I tried it in Yocto and it
> didn't work properly, not all the dependents were rebuilt. If it works,
> it's a very good feature!
Yes, I'm using it since this weekend for both Isar as well as
jailhouse-images development, and it helped a lot already. The only
limitation I found, but I do not remember right now if that isn't
inherent to bitbake, is that it does not detect changes in files that
the recipes carries in its SRC_URI (file://...). Then you need "-c
clean" for the affected target - which now works as well.
>
> Also do you know if this feature relies on DEPENDS variable? If so, we
> should keep this in mind with the idea to drop DEPENDS from recipes.
Don't ask me for the details, but it surely relies on task and target
dependencies, however they are calculated internally by bitbake.
Jan
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 4/6] Enable proper rebuilds on dependency changes
2018-02-13 16:21 ` Jan Kiszka
@ 2018-02-13 17:57 ` Jan Kiszka
2018-02-13 18:08 ` Alexander Smirnov
0 siblings, 1 reply; 27+ messages in thread
From: Jan Kiszka @ 2018-02-13 17:57 UTC (permalink / raw)
To: Alexander Smirnov, isar-users
On 2018-02-13 17:21, Jan Kiszka wrote:
> On 2018-02-13 14:03, Alexander Smirnov wrote:
>> On 02/12/2018 10:59 AM, Jan Kiszka wrote:
>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>>
>>> Install a basichash as signature handler and set the stamp policy to
>>> full - and suddenly we get proper rebuilds of the image and all affected
>>> packages when some recipe down the dependency chain changed!
>>>
>>> We are using the legacy bitbake mechanism here as we do not have
>>> setscene machinery like OE. Still good enough for us.
>>>
>>
>> BTW: have you tried this? Two years ago I tried it in Yocto and it
>> didn't work properly, not all the dependents were rebuilt. If it works,
>> it's a very good feature!
>
> Yes, I'm using it since this weekend for both Isar as well as
> jailhouse-images development, and it helped a lot already. The only
> limitation I found, but I do not remember right now if that isn't
> inherent to bitbake, is that it does not detect changes in files that
> the recipes carries in its SRC_URI (file://...). Then you need "-c
> clean" for the affected target - which now works as well.
>
Found another issue, which worked surprisingly well so far despite being
fairly broken: do_setup_mounts is not re-run on rebuilds.
Should be fixable, specifically by avoiding to stamp it.
Jan
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 4/6] Enable proper rebuilds on dependency changes
2018-02-13 17:57 ` Jan Kiszka
@ 2018-02-13 18:08 ` Alexander Smirnov
2018-02-13 18:44 ` Jan Kiszka
0 siblings, 1 reply; 27+ messages in thread
From: Alexander Smirnov @ 2018-02-13 18:08 UTC (permalink / raw)
To: Jan Kiszka, isar-users
On 02/13/2018 08:57 PM, Jan Kiszka wrote:
> On 2018-02-13 17:21, Jan Kiszka wrote:
>> On 2018-02-13 14:03, Alexander Smirnov wrote:
>>> On 02/12/2018 10:59 AM, Jan Kiszka wrote:
>>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>>>
>>>> Install a basichash as signature handler and set the stamp policy to
>>>> full - and suddenly we get proper rebuilds of the image and all affected
>>>> packages when some recipe down the dependency chain changed!
>>>>
>>>> We are using the legacy bitbake mechanism here as we do not have
>>>> setscene machinery like OE. Still good enough for us.
>>>>
>>>
>>> BTW: have you tried this? Two years ago I tried it in Yocto and it
>>> didn't work properly, not all the dependents were rebuilt. If it works,
>>> it's a very good feature!
>>
>> Yes, I'm using it since this weekend for both Isar as well as
>> jailhouse-images development, and it helped a lot already. The only
>> limitation I found, but I do not remember right now if that isn't
>> inherent to bitbake, is that it does not detect changes in files that
>> the recipes carries in its SRC_URI (file://...). Then you need "-c
>> clean" for the affected target - which now works as well.
>>
>
> Found another issue, which worked surprisingly well so far despite being
> fairly broken: do_setup_mounts is not re-run on rebuilds.
Investigating exactly this issue. It occures for clean builds also. My
guess is patch #3, currently I'm building with only 2 first applied.
With 3rd one applied build failed.
Alex
>
> Should be fixable, specifically by avoiding to stamp it.
>
> Jan
>
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 4/6] Enable proper rebuilds on dependency changes
2018-02-13 18:08 ` Alexander Smirnov
@ 2018-02-13 18:44 ` Jan Kiszka
2018-02-13 19:02 ` Alexander Smirnov
0 siblings, 1 reply; 27+ messages in thread
From: Jan Kiszka @ 2018-02-13 18:44 UTC (permalink / raw)
To: Alexander Smirnov, isar-users
On 2018-02-13 19:08, Alexander Smirnov wrote:
> On 02/13/2018 08:57 PM, Jan Kiszka wrote:
>> On 2018-02-13 17:21, Jan Kiszka wrote:
>>> On 2018-02-13 14:03, Alexander Smirnov wrote:
>>>> On 02/12/2018 10:59 AM, Jan Kiszka wrote:
>>>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>>>>
>>>>> Install a basichash as signature handler and set the stamp policy to
>>>>> full - and suddenly we get proper rebuilds of the image and all
>>>>> affected
>>>>> packages when some recipe down the dependency chain changed!
>>>>>
>>>>> We are using the legacy bitbake mechanism here as we do not have
>>>>> setscene machinery like OE. Still good enough for us.
>>>>>
>>>>
>>>> BTW: have you tried this? Two years ago I tried it in Yocto and it
>>>> didn't work properly, not all the dependents were rebuilt. If it works,
>>>> it's a very good feature!
>>>
>>> Yes, I'm using it since this weekend for both Isar as well as
>>> jailhouse-images development, and it helped a lot already. The only
>>> limitation I found, but I do not remember right now if that isn't
>>> inherent to bitbake, is that it does not detect changes in files that
>>> the recipes carries in its SRC_URI (file://...). Then you need "-c
>>> clean" for the affected target - which now works as well.
>>>
>>
>> Found another issue, which worked surprisingly well so far despite being
>> fairly broken: do_setup_mounts is not re-run on rebuilds.
>
> Investigating exactly this issue. It occures for clean builds also. My
> guess is patch #3, currently I'm building with only 2 first applied.
> With 3rd one applied build failed.
>
That's not the right approach, the concept of the setup_mounts task is
broken.
I'm still testing, but it looks like as if this massive simplification
can also fix the rebuild issue:
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index a45fbce..aa46f45 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -2,7 +2,7 @@
# Copyright (C) 2017 Siemens AG
# Add dependency from buildchroot creation
-do_build[depends] = "buildchroot:do_setup_mounts"
+do_build[depends] = "buildchroot:do_build"
# Add dependency between Isar recipes
DEPENDS ?= ""
@@ -25,6 +25,14 @@ do_build() {
mkdir -p ${BUILDROOT}
sudo mount --bind ${WORKDIR} ${BUILDROOT}
+ # These tests may race with parallel builds, so ignore errors.
+ grep -q ${BUILDCHROOT_DIR}/isar-apt /proc/mounts &&
+ sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} ${BUILDCHROOT_DIR}/isar-apt || true
+ grep -q ${BUILDCHROOT_DIR}/dev /proc/mounts &&
+ sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${BUILDCHROOT_DIR}/dev || true
+ grep -q ${BUILDCHROOT_DIR}/proc /proc/mounts &&
+ sudo mount -t proc none ${BUILDCHROOT_DIR}/proc || true
+
dpkg_runbuild
sudo umount ${BUILDROOT} 2>/dev/null || true
diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb
index 570c0ad..cd91228 100644
--- a/meta/recipes-devtools/buildchroot/buildchroot.bb
+++ b/meta/recipes-devtools/buildchroot/buildchroot.bb
@@ -60,7 +60,9 @@ do_build() {
-e 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \
"${WORKDIR}/multistrap.conf.in" > "${WORKDIR}/multistrap.conf"
- do_setup_mounts
+ sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} ${BUILDCHROOT_DIR}/isar-apt
+ sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${BUILDCHROOT_DIR}/dev
+ sudo mount -t proc none ${BUILDCHROOT_DIR}/proc
# Create root filesystem
sudo -E multistrap -a ${DISTRO_ARCH} -d "${BUILDCHROOT_DIR}" -f "${WORKDIR}/multistrap.conf"
@@ -70,30 +72,4 @@ do_build() {
# Configure root filesystem
sudo chroot ${BUILDCHROOT_DIR} /configscript.sh
-
- do_cleanup_mounts
-}
-
-# Invalidate stamp for do_setup_mounts before each build start.
-# This will guarantee that this function will be executed once
-# per build.
-python __anonymous() {
- stamp = d.getVar("STAMP") + ".do_setup_mounts." + d.getVarFlag("do_setup_mounts", 'stamp-extra-info')
- os.remove(stamp) if os.path.exists(stamp) else None
-}
-
-do_setup_mounts[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
-
-do_setup_mounts() {
- sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} ${BUILDCHROOT_DIR}/isar-apt
- sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${BUILDCHROOT_DIR}/dev
- sudo mount -t proc none ${BUILDCHROOT_DIR}/proc
-}
-
-addtask setup_mounts after do_build
-
-do_cleanup_mounts() {
- sudo umount ${BUILDCHROOT_DIR}/isar-apt 2>/dev/null || true
- sudo umount ${BUILDCHROOT_DIR}/dev 2>/dev/null || true
- sudo umount ${BUILDCHROOT_DIR}/proc 2>/dev/null || true
}
Jan
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 4/6] Enable proper rebuilds on dependency changes
2018-02-13 18:44 ` Jan Kiszka
@ 2018-02-13 19:02 ` Alexander Smirnov
2018-02-13 19:22 ` Jan Kiszka
0 siblings, 1 reply; 27+ messages in thread
From: Alexander Smirnov @ 2018-02-13 19:02 UTC (permalink / raw)
To: Jan Kiszka, isar-users
On 02/13/2018 09:44 PM, Jan Kiszka wrote:
> On 2018-02-13 19:08, Alexander Smirnov wrote:
>> On 02/13/2018 08:57 PM, Jan Kiszka wrote:
>>> On 2018-02-13 17:21, Jan Kiszka wrote:
>>>> On 2018-02-13 14:03, Alexander Smirnov wrote:
>>>>> On 02/12/2018 10:59 AM, Jan Kiszka wrote:
>>>>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>>>>>
>>>>>> Install a basichash as signature handler and set the stamp policy to
>>>>>> full - and suddenly we get proper rebuilds of the image and all
>>>>>> affected
>>>>>> packages when some recipe down the dependency chain changed!
>>>>>>
>>>>>> We are using the legacy bitbake mechanism here as we do not have
>>>>>> setscene machinery like OE. Still good enough for us.
>>>>>>
>>>>>
>>>>> BTW: have you tried this? Two years ago I tried it in Yocto and it
>>>>> didn't work properly, not all the dependents were rebuilt. If it works,
>>>>> it's a very good feature!
>>>>
>>>> Yes, I'm using it since this weekend for both Isar as well as
>>>> jailhouse-images development, and it helped a lot already. The only
>>>> limitation I found, but I do not remember right now if that isn't
>>>> inherent to bitbake, is that it does not detect changes in files that
>>>> the recipes carries in its SRC_URI (file://...). Then you need "-c
>>>> clean" for the affected target - which now works as well.
>>>>
>>>
>>> Found another issue, which worked surprisingly well so far despite being
>>> fairly broken: do_setup_mounts is not re-run on rebuilds.
>>
>> Investigating exactly this issue. It occures for clean builds also. My
>> guess is patch #3, currently I'm building with only 2 first applied.
>> With 3rd one applied build failed.
>>
>
> That's not the right approach, the concept of the setup_mounts task is
> broken.
Why you think it's broken? Just tested, it works good:
--- a/meta/recipes-devtools/buildchroot/buildchroot.bb
+++ b/meta/recipes-devtools/buildchroot/buildchroot.bb
@@ -88,6 +88,8 @@ do_setup_mounts() {
sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO}
${BUILDCHROOT_DIR}/isar-apt
sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs
${BUILDCHROOT_DIR}/dev
sudo mount -t proc none ${BUILDCHROOT_DIR}/proc
+
+ echo XXX > /tmp/test
}
Applying this patch I have /tmp/test on re-run.
>
> I'm still testing, but it looks like as if this massive simplification
> can also fix the rebuild issue:
The issue is with the patch#2, dropping it fixes the build. Probably the
issue is that do_clean has no before/after specifier in the pipeline.
Still testing.
Alex
>
> diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
> index a45fbce..aa46f45 100644
> --- a/meta/classes/dpkg-base.bbclass
> +++ b/meta/classes/dpkg-base.bbclass
> @@ -2,7 +2,7 @@
> # Copyright (C) 2017 Siemens AG
>
> # Add dependency from buildchroot creation
> -do_build[depends] = "buildchroot:do_setup_mounts"
> +do_build[depends] = "buildchroot:do_build"
>
> # Add dependency between Isar recipes
> DEPENDS ?= ""
> @@ -25,6 +25,14 @@ do_build() {
> mkdir -p ${BUILDROOT}
> sudo mount --bind ${WORKDIR} ${BUILDROOT}
>
> + # These tests may race with parallel builds, so ignore errors.
> + grep -q ${BUILDCHROOT_DIR}/isar-apt /proc/mounts &&
> + sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} ${BUILDCHROOT_DIR}/isar-apt || true
> + grep -q ${BUILDCHROOT_DIR}/dev /proc/mounts &&
> + sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${BUILDCHROOT_DIR}/dev || true
> + grep -q ${BUILDCHROOT_DIR}/proc /proc/mounts &&
> + sudo mount -t proc none ${BUILDCHROOT_DIR}/proc || true
> +
> dpkg_runbuild
>
> sudo umount ${BUILDROOT} 2>/dev/null || true
> diff --git a/meta/recipes-devtools/buildchroot/buildchroot.bb b/meta/recipes-devtools/buildchroot/buildchroot.bb
> index 570c0ad..cd91228 100644
> --- a/meta/recipes-devtools/buildchroot/buildchroot.bb
> +++ b/meta/recipes-devtools/buildchroot/buildchroot.bb
> @@ -60,7 +60,9 @@ do_build() {
> -e 's|##DIR_HOOKS##|./'"$WORKDIR_REL"'/hooks_multistrap|g' \
> "${WORKDIR}/multistrap.conf.in" > "${WORKDIR}/multistrap.conf"
>
> - do_setup_mounts
> + sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} ${BUILDCHROOT_DIR}/isar-apt
> + sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${BUILDCHROOT_DIR}/dev
> + sudo mount -t proc none ${BUILDCHROOT_DIR}/proc
>
> # Create root filesystem
> sudo -E multistrap -a ${DISTRO_ARCH} -d "${BUILDCHROOT_DIR}" -f "${WORKDIR}/multistrap.conf"
> @@ -70,30 +72,4 @@ do_build() {
>
> # Configure root filesystem
> sudo chroot ${BUILDCHROOT_DIR} /configscript.sh
> -
> - do_cleanup_mounts
> -}
> -
> -# Invalidate stamp for do_setup_mounts before each build start.
> -# This will guarantee that this function will be executed once
> -# per build.
> -python __anonymous() {
> - stamp = d.getVar("STAMP") + ".do_setup_mounts." + d.getVarFlag("do_setup_mounts", 'stamp-extra-info')
> - os.remove(stamp) if os.path.exists(stamp) else None
> -}
> -
> -do_setup_mounts[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> -
> -do_setup_mounts() {
> - sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO} ${BUILDCHROOT_DIR}/isar-apt
> - sudo mount -t devtmpfs -o mode=0755,nosuid devtmpfs ${BUILDCHROOT_DIR}/dev
> - sudo mount -t proc none ${BUILDCHROOT_DIR}/proc
> -}
> -
> -addtask setup_mounts after do_build
> -
> -do_cleanup_mounts() {
> - sudo umount ${BUILDCHROOT_DIR}/isar-apt 2>/dev/null || true
> - sudo umount ${BUILDCHROOT_DIR}/dev 2>/dev/null || true
> - sudo umount ${BUILDCHROOT_DIR}/proc 2>/dev/null || true
> }
>
>
> Jan
>
--
With best regards,
Alexander Smirnov
ilbers GmbH
Baierbrunner Str. 28c
D-81379 Munich
+49 (89) 122 67 24-0
http://ilbers.de/
Commercial register Munich, HRB 214197
General manager: Baurzhan Ismagulov
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 4/6] Enable proper rebuilds on dependency changes
2018-02-13 19:02 ` Alexander Smirnov
@ 2018-02-13 19:22 ` Jan Kiszka
2018-02-13 19:38 ` Alexander Smirnov
0 siblings, 1 reply; 27+ messages in thread
From: Jan Kiszka @ 2018-02-13 19:22 UTC (permalink / raw)
To: Alexander Smirnov, isar-users
On 2018-02-13 20:02, Alexander Smirnov wrote:
>
> On 02/13/2018 09:44 PM, Jan Kiszka wrote:
>> On 2018-02-13 19:08, Alexander Smirnov wrote:
>>> On 02/13/2018 08:57 PM, Jan Kiszka wrote:
>>>> On 2018-02-13 17:21, Jan Kiszka wrote:
>>>>> On 2018-02-13 14:03, Alexander Smirnov wrote:
>>>>>> On 02/12/2018 10:59 AM, Jan Kiszka wrote:
>>>>>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>>>>>>
>>>>>>> Install a basichash as signature handler and set the stamp policy to
>>>>>>> full - and suddenly we get proper rebuilds of the image and all
>>>>>>> affected
>>>>>>> packages when some recipe down the dependency chain changed!
>>>>>>>
>>>>>>> We are using the legacy bitbake mechanism here as we do not have
>>>>>>> setscene machinery like OE. Still good enough for us.
>>>>>>>
>>>>>>
>>>>>> BTW: have you tried this? Two years ago I tried it in Yocto and it
>>>>>> didn't work properly, not all the dependents were rebuilt. If it
>>>>>> works,
>>>>>> it's a very good feature!
>>>>>
>>>>> Yes, I'm using it since this weekend for both Isar as well as
>>>>> jailhouse-images development, and it helped a lot already. The only
>>>>> limitation I found, but I do not remember right now if that isn't
>>>>> inherent to bitbake, is that it does not detect changes in files that
>>>>> the recipes carries in its SRC_URI (file://...). Then you need "-c
>>>>> clean" for the affected target - which now works as well.
>>>>>
>>>>
>>>> Found another issue, which worked surprisingly well so far despite
>>>> being
>>>> fairly broken: do_setup_mounts is not re-run on rebuilds.
>>>
>>> Investigating exactly this issue. It occures for clean builds also. My
>>> guess is patch #3, currently I'm building with only 2 first applied.
>>> With 3rd one applied build failed.
>>>
>>
>> That's not the right approach, the concept of the setup_mounts task is
>> broken.
>
> Why you think it's broken? Just tested, it works good:
It's broken in its design: you stamp a task and then remove that stamp
again on restarts. That only works because we didn't generate the full
dependency chain. Once you do, all dependent tasks we be kicked for
re-run. Removing patch 3 is voodoo debugging. I bet you can't explain
why that works around the issue, and I'm sure it will break eventually
again. Moreover, it's needlessly complex.
The much simpler solution is to try-mount when needed, at user side.
Patch will follow, likely by resending the whole series
Jan
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 4/6] Enable proper rebuilds on dependency changes
2018-02-13 19:22 ` Jan Kiszka
@ 2018-02-13 19:38 ` Alexander Smirnov
2018-02-13 20:04 ` Jan Kiszka
0 siblings, 1 reply; 27+ messages in thread
From: Alexander Smirnov @ 2018-02-13 19:38 UTC (permalink / raw)
To: Jan Kiszka, isar-users
On 02/13/2018 10:22 PM, Jan Kiszka wrote:
> On 2018-02-13 20:02, Alexander Smirnov wrote:
>>
>> On 02/13/2018 09:44 PM, Jan Kiszka wrote:
>>> On 2018-02-13 19:08, Alexander Smirnov wrote:
>>>> On 02/13/2018 08:57 PM, Jan Kiszka wrote:
>>>>> On 2018-02-13 17:21, Jan Kiszka wrote:
>>>>>> On 2018-02-13 14:03, Alexander Smirnov wrote:
>>>>>>> On 02/12/2018 10:59 AM, Jan Kiszka wrote:
>>>>>>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>>>>>>>
>>>>>>>> Install a basichash as signature handler and set the stamp policy to
>>>>>>>> full - and suddenly we get proper rebuilds of the image and all
>>>>>>>> affected
>>>>>>>> packages when some recipe down the dependency chain changed!
>>>>>>>>
>>>>>>>> We are using the legacy bitbake mechanism here as we do not have
>>>>>>>> setscene machinery like OE. Still good enough for us.
>>>>>>>>
>>>>>>>
>>>>>>> BTW: have you tried this? Two years ago I tried it in Yocto and it
>>>>>>> didn't work properly, not all the dependents were rebuilt. If it
>>>>>>> works,
>>>>>>> it's a very good feature!
>>>>>>
>>>>>> Yes, I'm using it since this weekend for both Isar as well as
>>>>>> jailhouse-images development, and it helped a lot already. The only
>>>>>> limitation I found, but I do not remember right now if that isn't
>>>>>> inherent to bitbake, is that it does not detect changes in files that
>>>>>> the recipes carries in its SRC_URI (file://...). Then you need "-c
>>>>>> clean" for the affected target - which now works as well.
>>>>>>
>>>>>
>>>>> Found another issue, which worked surprisingly well so far despite
>>>>> being
>>>>> fairly broken: do_setup_mounts is not re-run on rebuilds.
>>>>
>>>> Investigating exactly this issue. It occures for clean builds also. My
>>>> guess is patch #3, currently I'm building with only 2 first applied.
>>>> With 3rd one applied build failed.
>>>>
>>>
>>> That's not the right approach, the concept of the setup_mounts task is
>>> broken.
>>
>> Why you think it's broken? Just tested, it works good:
>
> It's broken in its design: you stamp a task and then remove that stamp
> again on restarts. That only works because we didn't generate the full
> dependency chain. Once you do, all dependent tasks we be kicked for
> re-run. Removing patch 3 is voodoo debugging. I bet you can't explain
> why that works around the issue, and I'm sure it will break eventually
> again. Moreover, it's needlessly complex.
CI was green, I apply this series, CI became red, I've reset the head
patch by patch to get CI green again. After reset patch #2 CI became
green. Which voodoo you found here? This series brakes the build without
re-run.
>
> The much simpler solution is to try-mount when needed, at user side.
> Patch will follow, likely by resending the whole series
Mount when needed like you proposed could lead to lots of mounts on the
same folder, because do_build works in parallel.
The correct way in this case would be:
- keep do_mounts
- use [nostamp] = 1
- use [lockfiles] = "${BUILDCHROOT_DIR}/lock"
- mount folders like you proposed:
+ grep -q ${BUILDCHROOT_DIR}/isar-apt /proc/mounts &&
+ sudo mount --bind ${DEPLOY_DIR_APT}/${DISTRO}
${BUILDCHROOT_DIR}/isar-apt || true
Alex
>
> Jan
>
--
With best regards,
Alexander Smirnov
ilbers GmbH
Baierbrunner Str. 28c
D-81379 Munich
+49 (89) 122 67 24-0
http://ilbers.de/
Commercial register Munich, HRB 214197
General manager: Baurzhan Ismagulov
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 4/6] Enable proper rebuilds on dependency changes
2018-02-13 19:38 ` Alexander Smirnov
@ 2018-02-13 20:04 ` Jan Kiszka
0 siblings, 0 replies; 27+ messages in thread
From: Jan Kiszka @ 2018-02-13 20:04 UTC (permalink / raw)
To: Alexander Smirnov, isar-users
On 2018-02-13 20:38, Alexander Smirnov wrote:
>
>
> On 02/13/2018 10:22 PM, Jan Kiszka wrote:
>> On 2018-02-13 20:02, Alexander Smirnov wrote:
>>>
>>> On 02/13/2018 09:44 PM, Jan Kiszka wrote:
>>>> On 2018-02-13 19:08, Alexander Smirnov wrote:
>>>>> On 02/13/2018 08:57 PM, Jan Kiszka wrote:
>>>>>> On 2018-02-13 17:21, Jan Kiszka wrote:
>>>>>>> On 2018-02-13 14:03, Alexander Smirnov wrote:
>>>>>>>> On 02/12/2018 10:59 AM, Jan Kiszka wrote:
>>>>>>>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>>>>>>>>
>>>>>>>>> Install a basichash as signature handler and set the stamp
>>>>>>>>> policy to
>>>>>>>>> full - and suddenly we get proper rebuilds of the image and all
>>>>>>>>> affected
>>>>>>>>> packages when some recipe down the dependency chain changed!
>>>>>>>>>
>>>>>>>>> We are using the legacy bitbake mechanism here as we do not have
>>>>>>>>> setscene machinery like OE. Still good enough for us.
>>>>>>>>>
>>>>>>>>
>>>>>>>> BTW: have you tried this? Two years ago I tried it in Yocto and it
>>>>>>>> didn't work properly, not all the dependents were rebuilt. If it
>>>>>>>> works,
>>>>>>>> it's a very good feature!
>>>>>>>
>>>>>>> Yes, I'm using it since this weekend for both Isar as well as
>>>>>>> jailhouse-images development, and it helped a lot already. The only
>>>>>>> limitation I found, but I do not remember right now if that isn't
>>>>>>> inherent to bitbake, is that it does not detect changes in files
>>>>>>> that
>>>>>>> the recipes carries in its SRC_URI (file://...). Then you need "-c
>>>>>>> clean" for the affected target - which now works as well.
>>>>>>>
>>>>>>
>>>>>> Found another issue, which worked surprisingly well so far despite
>>>>>> being
>>>>>> fairly broken: do_setup_mounts is not re-run on rebuilds.
>>>>>
>>>>> Investigating exactly this issue. It occures for clean builds also. My
>>>>> guess is patch #3, currently I'm building with only 2 first applied.
>>>>> With 3rd one applied build failed.
>>>>>
>>>>
>>>> That's not the right approach, the concept of the setup_mounts task is
>>>> broken.
>>>
>>> Why you think it's broken? Just tested, it works good:
>>
>> It's broken in its design: you stamp a task and then remove that stamp
>> again on restarts. That only works because we didn't generate the full
>> dependency chain. Once you do, all dependent tasks we be kicked for
>> re-run. Removing patch 3 is voodoo debugging. I bet you can't explain
>> why that works around the issue, and I'm sure it will break eventually
>> again. Moreover, it's needlessly complex.
>
> CI was green, I apply this series, CI became red, I've reset the head
> patch by patch to get CI green again. After reset patch #2 CI became
> green. Which voodoo you found here? This series brakes the build without
> re-run.
>
>>
>> The much simpler solution is to try-mount when needed, at user side.
>> Patch will follow, likely by resending the whole series
>
> Mount when needed like you proposed could lead to lots of mounts on the
> same folder, because do_build works in parallel.
All solved, in a single line. Hold your breath, it's getting better.
Jan
^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH 5/6] dpkg-raw: Clean DEBIAN dir prior to filling it
2018-02-12 7:59 [PATCH 0/6] Add support for automatic partial rebuilds on recipe changes Jan Kiszka
` (3 preceding siblings ...)
2018-02-12 7:59 ` [PATCH 4/6] Enable proper rebuilds on dependency changes Jan Kiszka
@ 2018-02-12 7:59 ` Jan Kiszka
2018-02-13 14:06 ` Alexander Smirnov
2018-02-12 7:59 ` [PATCH 6/6] isar-image-base: Clean rootfs folder prior to building Jan Kiszka
` (2 subsequent siblings)
7 siblings, 1 reply; 27+ messages in thread
From: Jan Kiszka @ 2018-02-12 7:59 UTC (permalink / raw)
To: isar-users
From: Jan Kiszka <jan.kiszka@siemens.com>
When we run unprivileged, writing files in that directory will not work
after the first run because everything is root-owned at the end.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
meta/classes/dpkg-raw.bbclass | 5 ++---
1 file changed, 2 insertions(+), 3 deletions(-)
diff --git a/meta/classes/dpkg-raw.bbclass b/meta/classes/dpkg-raw.bbclass
index 44aa078..e3ae9e8 100644
--- a/meta/classes/dpkg-raw.bbclass
+++ b/meta/classes/dpkg-raw.bbclass
@@ -16,10 +16,9 @@ do_install() {
do_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
addtask install after do_unpack before do_deb_package_prepare
-# so we can put hooks etc. in there already
-do_install[dirs] = "${D}/DEBIAN"
-
do_deb_package_prepare() {
+ sudo rm -rf ${D}/DEBIAN
+ mkdir ${D}/DEBIAN
cat<<-__EOF__ > ${D}/DEBIAN/control
Package: ${PN}
Architecture: ${DISTRO_ARCH}
--
2.13.6
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 5/6] dpkg-raw: Clean DEBIAN dir prior to filling it
2018-02-12 7:59 ` [PATCH 5/6] dpkg-raw: Clean DEBIAN dir prior to filling it Jan Kiszka
@ 2018-02-13 14:06 ` Alexander Smirnov
2018-02-13 16:22 ` Jan Kiszka
0 siblings, 1 reply; 27+ messages in thread
From: Alexander Smirnov @ 2018-02-13 14:06 UTC (permalink / raw)
To: Jan Kiszka, isar-users
On 02/12/2018 10:59 AM, Jan Kiszka wrote:
> From: Jan Kiszka <jan.kiszka@siemens.com>
>
> When we run unprivileged, writing files in that directory will not work
> after the first run because everything is root-owned at the end.
>
> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
> meta/classes/dpkg-raw.bbclass | 5 ++---
> 1 file changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/meta/classes/dpkg-raw.bbclass b/meta/classes/dpkg-raw.bbclass
> index 44aa078..e3ae9e8 100644
> --- a/meta/classes/dpkg-raw.bbclass
> +++ b/meta/classes/dpkg-raw.bbclass
> @@ -16,10 +16,9 @@ do_install() {
> do_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
> addtask install after do_unpack before do_deb_package_prepare
>
> -# so we can put hooks etc. in there already
> -do_install[dirs] = "${D}/DEBIAN"
Here you remove this folder, while it's used in example-raw recipe.
Alex
> -
> do_deb_package_prepare() {
> + sudo rm -rf ${D}/DEBIAN
> + mkdir ${D}/DEBIAN
> cat<<-__EOF__ > ${D}/DEBIAN/control
> Package: ${PN}
> Architecture: ${DISTRO_ARCH}
>
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 5/6] dpkg-raw: Clean DEBIAN dir prior to filling it
2018-02-13 14:06 ` Alexander Smirnov
@ 2018-02-13 16:22 ` Jan Kiszka
2018-02-13 16:31 ` Alexander Smirnov
0 siblings, 1 reply; 27+ messages in thread
From: Jan Kiszka @ 2018-02-13 16:22 UTC (permalink / raw)
To: Alexander Smirnov, isar-users
On 2018-02-13 15:06, Alexander Smirnov wrote:
>
>
> On 02/12/2018 10:59 AM, Jan Kiszka wrote:
>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>
>> When we run unprivileged, writing files in that directory will not work
>> after the first run because everything is root-owned at the end.
>>
>> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
>> ---
>> meta/classes/dpkg-raw.bbclass | 5 ++---
>> 1 file changed, 2 insertions(+), 3 deletions(-)
>>
>> diff --git a/meta/classes/dpkg-raw.bbclass
>> b/meta/classes/dpkg-raw.bbclass
>> index 44aa078..e3ae9e8 100644
>> --- a/meta/classes/dpkg-raw.bbclass
>> +++ b/meta/classes/dpkg-raw.bbclass
>> @@ -16,10 +16,9 @@ do_install() {
>> do_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>> addtask install after do_unpack before do_deb_package_prepare
>> -# so we can put hooks etc. in there already
>> -do_install[dirs] = "${D}/DEBIAN"
>
> Here you remove this folder, while it's used in example-raw recipe.
> > Alex
>
>> -
>> do_deb_package_prepare() {
>> + sudo rm -rf ${D}/DEBIAN
>> + mkdir ${D}/DEBIAN
It is created here now - nothing is lost. If you tell me a good reason
to keep it listed above as well, I'm fine with leaving that line alone.
Jan
>> cat<<-__EOF__ > ${D}/DEBIAN/control
>> Package: ${PN}
>> Architecture: ${DISTRO_ARCH}
>>
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 5/6] dpkg-raw: Clean DEBIAN dir prior to filling it
2018-02-13 16:22 ` Jan Kiszka
@ 2018-02-13 16:31 ` Alexander Smirnov
2018-02-13 16:33 ` Jan Kiszka
0 siblings, 1 reply; 27+ messages in thread
From: Alexander Smirnov @ 2018-02-13 16:31 UTC (permalink / raw)
To: Jan Kiszka, isar-users
Jan Kiszka <jan.kiszka@siemens.com> 13 февраля 2018 г. 19:22:23 написал:
> On 2018-02-13 15:06, Alexander Smirnov wrote:
>>
>>
>> On 02/12/2018 10:59 AM, Jan Kiszka wrote:
>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>>
>>> When we run unprivileged, writing files in that directory will not work
>>> after the first run because everything is root-owned at the end.
>>>
>>> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
>>> ---
>>> meta/classes/dpkg-raw.bbclass | 5 ++---
>>> 1 file changed, 2 insertions(+), 3 deletions(-)
>>>
>>> diff --git a/meta/classes/dpkg-raw.bbclass
>>> b/meta/classes/dpkg-raw.bbclass
>>> index 44aa078..e3ae9e8 100644
>>> --- a/meta/classes/dpkg-raw.bbclass
>>> +++ b/meta/classes/dpkg-raw.bbclass
>>> @@ -16,10 +16,9 @@ do_install() {
>>> do_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>>> addtask install after do_unpack before do_deb_package_prepare
>>> -# so we can put hooks etc. in there already
>>> -do_install[dirs] = "${D}/DEBIAN"
>>
>> Here you remove this folder, while it's used in example-raw recipe.
I mean here you break the build for next and master.
Alex
>> > Alex
>>
>>> -
>>> do_deb_package_prepare() {
>>> + sudo rm -rf ${D}/DEBIAN
>>> + mkdir ${D}/DEBIAN
>
> It is created here now - nothing is lost. If you tell me a good reason
> to keep it listed above as well, I'm fine with leaving that line alone.
>
> Jan
>
>>> cat<<-__EOF__ > ${D}/DEBIAN/control
>>> Package: ${PN}
>>> Architecture: ${DISTRO_ARCH}
>>>
>
--
With best regards,
Alexander Smirnov
ilbers GmbH
Baierbrunner Str. 28c
D-81379 Munich
+49 (89) 122 67 24-0
http://ilbers.de/
Commercial register Munich, HRB 214197
General manager: Baurzhan Ismagulov
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 5/6] dpkg-raw: Clean DEBIAN dir prior to filling it
2018-02-13 16:31 ` Alexander Smirnov
@ 2018-02-13 16:33 ` Jan Kiszka
0 siblings, 0 replies; 27+ messages in thread
From: Jan Kiszka @ 2018-02-13 16:33 UTC (permalink / raw)
To: Alexander Smirnov, isar-users
On 2018-02-13 17:31, Alexander Smirnov wrote:
>
>
>
> Jan Kiszka <jan.kiszka@siemens.com> 13 февраля 2018 г. 19:22:23 написал:
>
>> On 2018-02-13 15:06, Alexander Smirnov wrote:
>>>
>>>
>>> On 02/12/2018 10:59 AM, Jan Kiszka wrote:
>>>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>>>
>>>> When we run unprivileged, writing files in that directory will not work
>>>> after the first run because everything is root-owned at the end.
>>>>
>>>> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
>>>> ---
>>>> meta/classes/dpkg-raw.bbclass | 5 ++---
>>>> 1 file changed, 2 insertions(+), 3 deletions(-)
>>>>
>>>> diff --git a/meta/classes/dpkg-raw.bbclass
>>>> b/meta/classes/dpkg-raw.bbclass
>>>> index 44aa078..e3ae9e8 100644
>>>> --- a/meta/classes/dpkg-raw.bbclass
>>>> +++ b/meta/classes/dpkg-raw.bbclass
>>>> @@ -16,10 +16,9 @@ do_install() {
>>>> do_install[stamp-extra-info] = "${DISTRO}-${DISTRO_ARCH}"
>>>> addtask install after do_unpack before do_deb_package_prepare
>>>> -# so we can put hooks etc. in there already
>>>> -do_install[dirs] = "${D}/DEBIAN"
>>>
>>> Here you remove this folder, while it's used in example-raw recipe.
>
> I mean here you break the build for next and master.
Yes, that is because my "dpkg-raw: Install hooks automatically" patch
has to be applied first (which prevents that the do_install step messes
with DEBIAN). Then the reasoning below applies.
Jan
>
> Alex
>
>>> > Alex
>>>
>>>> -
>>>> do_deb_package_prepare() {
>>>> + sudo rm -rf ${D}/DEBIAN
>>>> + mkdir ${D}/DEBIAN
>>
>> It is created here now - nothing is lost. If you tell me a good reason
>> to keep it listed above as well, I'm fine with leaving that line alone.
>>
>> Jan
>>
>>>> cat<<-__EOF__ > ${D}/DEBIAN/control
>>>> Package: ${PN}
>>>> Architecture: ${DISTRO_ARCH}
>>>>
>>
> --
> With best regards,
> Alexander Smirnov
>
> ilbers GmbH
> Baierbrunner Str. 28c
> D-81379 Munich
> +49 (89) 122 67 24-0
> http://ilbers.de/
> Commercial register Munich, HRB 214197
> General manager: Baurzhan Ismagulov
>
>
^ permalink raw reply [flat|nested] 27+ messages in thread
* [PATCH 6/6] isar-image-base: Clean rootfs folder prior to building
2018-02-12 7:59 [PATCH 0/6] Add support for automatic partial rebuilds on recipe changes Jan Kiszka
` (4 preceding siblings ...)
2018-02-12 7:59 ` [PATCH 5/6] dpkg-raw: Clean DEBIAN dir prior to filling it Jan Kiszka
@ 2018-02-12 7:59 ` Jan Kiszka
2018-02-13 13:49 ` Alexander Smirnov
2018-02-13 7:40 ` [PATCH 0/6] Add support for automatic partial rebuilds on recipe changes Jan Kiszka
2018-02-13 14:01 ` Alexander Smirnov
7 siblings, 1 reply; 27+ messages in thread
From: Jan Kiszka @ 2018-02-12 7:59 UTC (permalink / raw)
To: isar-users
From: Jan Kiszka <jan.kiszka@siemens.com>
This ensures we have a clean base when rerunning the task.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
meta-isar/recipes-core/images/isar-image-base.bb | 2 ++
1 file changed, 2 insertions(+)
diff --git a/meta-isar/recipes-core/images/isar-image-base.bb b/meta-isar/recipes-core/images/isar-image-base.bb
index 2d16f8f..3d441cc 100644
--- a/meta-isar/recipes-core/images/isar-image-base.bb
+++ b/meta-isar/recipes-core/images/isar-image-base.bb
@@ -33,6 +33,8 @@ do_rootfs[dirs] = "${WORKDIR}/hooks_multistrap"
do_rootfs() {
E="${@ bb.utils.export_proxies(d)}"
+ sudo rm -rf ${IMAGE_ROOTFS}
+
chmod +x "${WORKDIR}/${DISTRO_CONFIG_SCRIPT}"
chmod +x "${WORKDIR}/setup.sh"
install -m 755 "${WORKDIR}/download_dev-random" "${WORKDIR}/hooks_multistrap/"
--
2.13.6
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 6/6] isar-image-base: Clean rootfs folder prior to building
2018-02-12 7:59 ` [PATCH 6/6] isar-image-base: Clean rootfs folder prior to building Jan Kiszka
@ 2018-02-13 13:49 ` Alexander Smirnov
2018-02-13 16:24 ` Jan Kiszka
0 siblings, 1 reply; 27+ messages in thread
From: Alexander Smirnov @ 2018-02-13 13:49 UTC (permalink / raw)
To: Jan Kiszka, isar-users
On 02/12/2018 10:59 AM, Jan Kiszka wrote:
> From: Jan Kiszka <jan.kiszka@siemens.com>
>
> This ensures we have a clean base when rerunning the task.
>
How do you do this?
$ bitbake -C do_rootfs ...?
If so, is it correct way to do this, I mean without do_clean before?
Alex
> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
> meta-isar/recipes-core/images/isar-image-base.bb | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/meta-isar/recipes-core/images/isar-image-base.bb b/meta-isar/recipes-core/images/isar-image-base.bb
> index 2d16f8f..3d441cc 100644
> --- a/meta-isar/recipes-core/images/isar-image-base.bb
> +++ b/meta-isar/recipes-core/images/isar-image-base.bb
> @@ -33,6 +33,8 @@ do_rootfs[dirs] = "${WORKDIR}/hooks_multistrap"
> do_rootfs() {
> E="${@ bb.utils.export_proxies(d)}"
>
> + sudo rm -rf ${IMAGE_ROOTFS}
> +
> chmod +x "${WORKDIR}/${DISTRO_CONFIG_SCRIPT}"
> chmod +x "${WORKDIR}/setup.sh"
> install -m 755 "${WORKDIR}/download_dev-random" "${WORKDIR}/hooks_multistrap/"
>
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 6/6] isar-image-base: Clean rootfs folder prior to building
2018-02-13 13:49 ` Alexander Smirnov
@ 2018-02-13 16:24 ` Jan Kiszka
0 siblings, 0 replies; 27+ messages in thread
From: Jan Kiszka @ 2018-02-13 16:24 UTC (permalink / raw)
To: Alexander Smirnov, isar-users
On 2018-02-13 14:49, Alexander Smirnov wrote:
> On 02/12/2018 10:59 AM, Jan Kiszka wrote:
>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>
>> This ensures we have a clean base when rerunning the task.
>>
>
> How do you do this?
>
> $ bitbake -C do_rootfs ...?
>
> If so, is it correct way to do this, I mean without do_clean before?
The task is rerun by bitbake now whenever it detects that some
dependency changed and had to be rerun as well. Or of the image recipe
itself got some changes. I do not need explicit "-c clean" in most cases
(see the other reply on patch 4).
Jan
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 27+ messages in thread
* Re: [PATCH 0/6] Add support for automatic partial rebuilds on recipe changes
2018-02-12 7:59 [PATCH 0/6] Add support for automatic partial rebuilds on recipe changes Jan Kiszka
` (5 preceding siblings ...)
2018-02-12 7:59 ` [PATCH 6/6] isar-image-base: Clean rootfs folder prior to building Jan Kiszka
@ 2018-02-13 7:40 ` Jan Kiszka
2018-02-13 14:01 ` Alexander Smirnov
7 siblings, 0 replies; 27+ messages in thread
From: Jan Kiszka @ 2018-02-13 7:40 UTC (permalink / raw)
To: isar-users, Alexander Smirnov
On 2018-02-12 08:59, [ext] Jan Kiszka wrote:
> Yeah, finally Yocto/OE-like usability: This ensures for many cases that
> changes to recipes lead to rebuilds of dependent recipes, including the
> final image. Some extra measures are needed so that those rebuilds work
> with clean dirs.
>
> And if the change detection should not work, e.g. changes to file://
> resources are not detected, then a clean or cleanall task is now
> available and ensures a proper manual rebuild.
>
> This massively increases the fun factor when developing Isar projects.
>
BTW, this series is independent of my kernel work. I just reordered both
in my internal queue so that the rework of the latter is more convenient
thanks to this one. Feel free to pull it first, unless you have remarks.
Jan
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 0/6] Add support for automatic partial rebuilds on recipe changes
2018-02-12 7:59 [PATCH 0/6] Add support for automatic partial rebuilds on recipe changes Jan Kiszka
` (6 preceding siblings ...)
2018-02-13 7:40 ` [PATCH 0/6] Add support for automatic partial rebuilds on recipe changes Jan Kiszka
@ 2018-02-13 14:01 ` Alexander Smirnov
2018-02-13 16:28 ` Jan Kiszka
7 siblings, 1 reply; 27+ messages in thread
From: Alexander Smirnov @ 2018-02-13 14:01 UTC (permalink / raw)
To: Jan Kiszka, isar-users
On 02/12/2018 10:59 AM, Jan Kiszka wrote:
> Yeah, finally Yocto/OE-like usability: This ensures for many cases that
> changes to recipes lead to rebuilds of dependent recipes, including the
> final image. Some extra measures are needed so that those rebuilds work
> with clean dirs.
>
> And if the change detection should not work, e.g. changes to file://
> resources are not detected, then a clean or cleanall task is now
> available and ensures a proper manual rebuild.
>
> This massively increases the fun factor when developing Isar projects.
>
Got error during build:
$ time bitbake multiconfig:qemuarm-wheezy:isar-image-base
multiconfig:qemuarm-jessie:isar-image-base
multiconfig:qemuarm-stretch:isar-image-base
multiconfig:qemui386-jessie:isar-image-base
multiconfig:qemui386-stretch:isar-image-base
multiconfig:qemuamd64-jessie:isar-image-base
multiconfig:qemuamd64-stretch:isar-image-base
multiconfig:rpi-jessie:isar-image-base
.........
builder@zbook:~/isar$ cat
/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/temp/log.do_install.16775
DEBUG: Executing shell function do_install
NOTE: Creating example-raw binary
NOTE: Putting example-raw into package
install: creating directory
‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/image’
install: creating directory
‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/image//usr’
install: creating directory
‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/image//usr/local’
install: creating directory
‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/image//usr/local/bin/’
‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/example-raw’
->
‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/image//usr/local/bin/example-raw’
NOTE: Now copy /README into package
install: creating directory
‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/image//usr/local/doc/’
‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/README’
->
‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/image//usr/local/doc/README-example-raw-0.2’
NOTE: Now for a debian hook, see dpkg-deb
‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/postinst’
->
‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/image//DEBIAN/postinst’
install: cannot create regular file
‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/image//DEBIAN/postinst’:
No such file or directory
WARNING: exit code 1 from a shell command.
ERROR: Function failed: do_install (log file is located at
/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/temp/log.do_install.16775)
Alex
^ permalink raw reply [flat|nested] 27+ messages in thread* Re: [PATCH 0/6] Add support for automatic partial rebuilds on recipe changes
2018-02-13 14:01 ` Alexander Smirnov
@ 2018-02-13 16:28 ` Jan Kiszka
0 siblings, 0 replies; 27+ messages in thread
From: Jan Kiszka @ 2018-02-13 16:28 UTC (permalink / raw)
To: Alexander Smirnov, isar-users
On 2018-02-13 15:01, Alexander Smirnov wrote:
> On 02/12/2018 10:59 AM, Jan Kiszka wrote:
>> Yeah, finally Yocto/OE-like usability: This ensures for many cases that
>> changes to recipes lead to rebuilds of dependent recipes, including the
>> final image. Some extra measures are needed so that those rebuilds work
>> with clean dirs.
>>
>> And if the change detection should not work, e.g. changes to file://
>> resources are not detected, then a clean or cleanall task is now
>> available and ensures a proper manual rebuild.
>>
>> This massively increases the fun factor when developing Isar projects.
>>
>
> Got error during build:
>
> $ time bitbake multiconfig:qemuarm-wheezy:isar-image-base
> multiconfig:qemuarm-jessie:isar-image-base
> multiconfig:qemuarm-stretch:isar-image-base
> multiconfig:qemui386-jessie:isar-image-base
> multiconfig:qemui386-stretch:isar-image-base
> multiconfig:qemuamd64-jessie:isar-image-base
> multiconfig:qemuamd64-stretch:isar-image-base
> multiconfig:rpi-jessie:isar-image-base
>
>
> .........
>
> builder@zbook:~/isar$ cat
> /home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/temp/log.do_install.16775
>
> DEBUG: Executing shell function do_install
> NOTE: Creating example-raw binary
> NOTE: Putting example-raw into package
> install: creating directory
> ‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/image’
>
> install: creating directory
> ‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/image//usr’
>
> install: creating directory
> ‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/image//usr/local’
>
> install: creating directory
> ‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/image//usr/local/bin/’
>
> ‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/example-raw’
> ->
> ‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/image//usr/local/bin/example-raw’
>
> NOTE: Now copy /README into package
> install: creating directory
> ‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/image//usr/local/doc/’
>
> ‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/README’
> ->
> ‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/image//usr/local/doc/README-example-raw-0.2’
>
> NOTE: Now for a debian hook, see dpkg-deb
> ‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/postinst’
> ->
> ‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/image//DEBIAN/postinst’
>
> install: cannot create regular file
> ‘/home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/image//DEBIAN/postinst’:
> No such file or directory
> WARNING: exit code 1 from a shell command.
> ERROR: Function failed: do_install (log file is located at
> /home/builder/isar/build/tmp/work/raspbian-jessie-armhf/example-raw-0.2-r0/temp/log.do_install.16775)
>
Oh! Yes, there is likely a dependency on "dpkg-raw: Install hooks
automatically" in this series, sorry. Please make sure that patch is
applied as well.
Jan
--
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 27+ messages in thread