* [PATCH 0/3] Make adjust_git work in both worlds
@ 2021-08-12 9:23 Jan Kiszka
2021-08-12 9:23 ` [PATCH 1/3] dpkg-base: Lock do_adjust_git against each other Jan Kiszka
` (2 more replies)
0 siblings, 3 replies; 5+ messages in thread
From: Jan Kiszka @ 2021-08-12 9:23 UTC (permalink / raw)
To: isar-users; +Cc: Uladzimir Bely, Henning Schild
Finally allow the adjusted git repos work both inside and outside the
buildchroots. Also fixes the race introduced by patching links in
DL_DIR.
NOTE: Not yet fully tested, our CI is busy with that now but just
stumbled again over the broken umount patches.
Jan
Jan Kiszka (3):
dpkg-base: Lock do_adjust_git against each other
dpkg-base: Make mirror link relative
Rework do_adjust_git to support inside and outside usage
meta/classes/dpkg-base.bbclass | 15 ++++++++++++---
meta/conf/bitbake.conf | 1 +
meta/recipes-devtools/buildchroot/buildchroot.inc | 2 ++
3 files changed, 15 insertions(+), 3 deletions(-)
--
2.31.1
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 1/3] dpkg-base: Lock do_adjust_git against each other
2021-08-12 9:23 [PATCH 0/3] Make adjust_git work in both worlds Jan Kiszka
@ 2021-08-12 9:23 ` Jan Kiszka
2021-08-12 9:23 ` [PATCH 2/3] dpkg-base: Make mirror link relative Jan Kiszka
2021-08-12 9:23 ` [PATCH 3/3] Rework do_adjust_git to support inside and outside usage Jan Kiszka
2 siblings, 0 replies; 5+ messages in thread
From: Jan Kiszka @ 2021-08-12 9:23 UTC (permalink / raw)
To: isar-users; +Cc: Uladzimir Bely, Henning Schild
From: Jan Kiszka <jan.kiszka@siemens.com>
This is needed since e47ada143469 because we now adjust the links to
mirrors in DL_DIR which may collide with other tasks doing this at the
same time.
Locking with also be needed soon for establishing the host-side link
to the download dir.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
meta/classes/dpkg-base.bbclass | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index ec8fbc1..fe16846 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -55,6 +55,7 @@ python do_adjust_git() {
}
addtask adjust_git before do_dpkg_build
+do_adjust_git[lockfiles] += "${DL_DIR}/isar.lock"
inherit patch
addtask patch before do_adjust_git
--
2.31.1
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 2/3] dpkg-base: Make mirror link relative
2021-08-12 9:23 [PATCH 0/3] Make adjust_git work in both worlds Jan Kiszka
2021-08-12 9:23 ` [PATCH 1/3] dpkg-base: Lock do_adjust_git against each other Jan Kiszka
@ 2021-08-12 9:23 ` Jan Kiszka
2021-08-12 9:23 ` [PATCH 3/3] Rework do_adjust_git to support inside and outside usage Jan Kiszka
2 siblings, 0 replies; 5+ messages in thread
From: Jan Kiszka @ 2021-08-12 9:23 UTC (permalink / raw)
To: isar-users; +Cc: Uladzimir Bely, Henning Schild
From: Jan Kiszka <jan.kiszka@siemens.com>
This will allow to use it both from the outer builder context and inside
the buildchroot. No idea why bitbake doesn't to that in the first place.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
meta/classes/dpkg-base.bbclass | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index fe16846..a4c8663 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -28,8 +28,10 @@ python do_adjust_git() {
if os.path.islink(ud.localpath):
realpath = os.path.realpath(ud.localpath)
- if realpath.startswith(d.getVar("DL_DIR")):
- link = realpath.replace(d.getVar("DL_DIR"), '/downloads', 1)
+ filter_out = os.path.join(d.getVar("DL_DIR"), "git") + "/"
+ if realpath.startswith(filter_out):
+ # make the link relative
+ link = realpath.replace(filter_out, '', 1)
os.unlink(ud.localpath)
os.symlink(link, ud.localpath)
--
2.31.1
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PATCH 3/3] Rework do_adjust_git to support inside and outside usage
2021-08-12 9:23 [PATCH 0/3] Make adjust_git work in both worlds Jan Kiszka
2021-08-12 9:23 ` [PATCH 1/3] dpkg-base: Lock do_adjust_git against each other Jan Kiszka
2021-08-12 9:23 ` [PATCH 2/3] dpkg-base: Make mirror link relative Jan Kiszka
@ 2021-08-12 9:23 ` Jan Kiszka
2021-08-12 10:31 ` Jan Kiszka
2 siblings, 1 reply; 5+ messages in thread
From: Jan Kiszka @ 2021-08-12 9:23 UTC (permalink / raw)
To: isar-users; +Cc: Uladzimir Bely, Henning Schild
From: Jan Kiszka <jan.kiszka@siemens.com>
Rewrite the alternates entry to use a relative path outside of the
workdir. That linke can then be created independently of the outer build
environment as well as inside the buildchroot.
We use ${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/.git-downloads outside
and /home/.git-downloads inside the buildchroot.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
meta/classes/dpkg-base.bbclass | 8 +++++++-
meta/conf/bitbake.conf | 1 +
meta/recipes-devtools/buildchroot/buildchroot.inc | 2 ++
3 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index a4c8663..cd6c649 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -19,6 +19,12 @@ python do_adjust_git() {
rootdir = d.getVar('WORKDIR', True)
+ git_link = os.path.join(d.getVar('GIT_DL_LINK_DIR'), '.git-downloads')
+ git_dl = os.path.join(d.getVar("DL_DIR"), "git")
+
+ if not os.path.exists(git_link) or os.path.realpath(git_link) != git_dl:
+ os.symlink(git_dl, git_link)
+
for src_uri in (d.getVar("SRC_URI", True) or "").split():
try:
fetcher = bb.fetch2.Fetch([src_uri], d)
@@ -48,7 +54,7 @@ python do_adjust_git() {
if os.path.exists(alternates):
cmd = ["sed", "-i", alternates, "-e",
- "s|{}|/downloads|".format(d.getVar("DL_DIR"))]
+ "s|{}|../../../../../.git-downloads|".format(git_dl)]
bb.note(' '.join(cmd))
if subprocess.call(cmd) != 0:
bb.fatal("git alternates adjustment failed")
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 9859456..7f5901d 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -48,6 +48,7 @@ GITPKGV = "${@bb.fetch2.get_srcrev(d, 'gitpkgv_revision')}"
# isar specific config
WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}/${PV}-${PR}"
+GIT_DL_LINK_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}"
DEPLOY_DIR_BOOTSTRAP = "${DEPLOY_DIR}/bootstrap"
DEPLOY_DIR_BUILDCHROOT = "${DEPLOY_DIR}/buildchroot"
DEPLOY_DIR_SDKCHROOT = "${DEPLOY_DIR}/sdkchroot"
diff --git a/meta/recipes-devtools/buildchroot/buildchroot.inc b/meta/recipes-devtools/buildchroot/buildchroot.inc
index 0c1fb7f..52a5f1c 100644
--- a/meta/recipes-devtools/buildchroot/buildchroot.inc
+++ b/meta/recipes-devtools/buildchroot/buildchroot.inc
@@ -67,6 +67,8 @@ buildchroot_install_files() {
sudo install -m 755 ${WORKDIR}/common.sh ${BUILDCHROOT_DIR}/isar/
sudo install -m 755 ${WORKDIR}/deps.sh ${BUILDCHROOT_DIR}/isar/
+ sudo ln -s /downloads/git "${BUILDCHROOT_DIR}/home/.git-downloads"
+
# Configure root filesystem
sudo install -m 755 ${WORKDIR}/configscript.sh ${BUILDCHROOT_DIR}
USER_ID=$(id -u)
--
2.31.1
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [PATCH 3/3] Rework do_adjust_git to support inside and outside usage
2021-08-12 9:23 ` [PATCH 3/3] Rework do_adjust_git to support inside and outside usage Jan Kiszka
@ 2021-08-12 10:31 ` Jan Kiszka
0 siblings, 0 replies; 5+ messages in thread
From: Jan Kiszka @ 2021-08-12 10:31 UTC (permalink / raw)
To: isar-users; +Cc: Uladzimir Bely, Henning Schild
On 12.08.21 11:23, Jan Kiszka wrote:
> From: Jan Kiszka <jan.kiszka@siemens.com>
>
> Rewrite the alternates entry to use a relative path outside of the
> workdir. That linke can then be created independently of the outer build
> environment as well as inside the buildchroot.
>
> We use ${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/.git-downloads outside
> and /home/.git-downloads inside the buildchroot.
>
> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
> meta/classes/dpkg-base.bbclass | 8 +++++++-
> meta/conf/bitbake.conf | 1 +
> meta/recipes-devtools/buildchroot/buildchroot.inc | 2 ++
> 3 files changed, 10 insertions(+), 1 deletion(-)
>
> diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
> index a4c8663..cd6c649 100644
> --- a/meta/classes/dpkg-base.bbclass
> +++ b/meta/classes/dpkg-base.bbclass
> @@ -19,6 +19,12 @@ python do_adjust_git() {
>
> rootdir = d.getVar('WORKDIR', True)
>
> + git_link = os.path.join(d.getVar('GIT_DL_LINK_DIR'), '.git-downloads')
> + git_dl = os.path.join(d.getVar("DL_DIR"), "git")
> +
> + if not os.path.exists(git_link) or os.path.realpath(git_link) != git_dl:
> + os.symlink(git_dl, git_link)
> +
> for src_uri in (d.getVar("SRC_URI", True) or "").split():
> try:
> fetcher = bb.fetch2.Fetch([src_uri], d)
> @@ -48,7 +54,7 @@ python do_adjust_git() {
>
> if os.path.exists(alternates):
> cmd = ["sed", "-i", alternates, "-e",
> - "s|{}|/downloads|".format(d.getVar("DL_DIR"))]
> + "s|{}|../../../../../.git-downloads|".format(git_dl)]
Hmm, this does not yet account for the case that the repo is placed in
some subdir under workdir. Will make this more general.
I also forgot to fix the task dependencies - devshell is not pulling
adjust_git.
Jan
> bb.note(' '.join(cmd))
> if subprocess.call(cmd) != 0:
> bb.fatal("git alternates adjustment failed")
> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
> index 9859456..7f5901d 100644
> --- a/meta/conf/bitbake.conf
> +++ b/meta/conf/bitbake.conf
> @@ -48,6 +48,7 @@ GITPKGV = "${@bb.fetch2.get_srcrev(d, 'gitpkgv_revision')}"
>
> # isar specific config
> WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}/${PV}-${PR}"
> +GIT_DL_LINK_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}"
> DEPLOY_DIR_BOOTSTRAP = "${DEPLOY_DIR}/bootstrap"
> DEPLOY_DIR_BUILDCHROOT = "${DEPLOY_DIR}/buildchroot"
> DEPLOY_DIR_SDKCHROOT = "${DEPLOY_DIR}/sdkchroot"
> diff --git a/meta/recipes-devtools/buildchroot/buildchroot.inc b/meta/recipes-devtools/buildchroot/buildchroot.inc
> index 0c1fb7f..52a5f1c 100644
> --- a/meta/recipes-devtools/buildchroot/buildchroot.inc
> +++ b/meta/recipes-devtools/buildchroot/buildchroot.inc
> @@ -67,6 +67,8 @@ buildchroot_install_files() {
> sudo install -m 755 ${WORKDIR}/common.sh ${BUILDCHROOT_DIR}/isar/
> sudo install -m 755 ${WORKDIR}/deps.sh ${BUILDCHROOT_DIR}/isar/
>
> + sudo ln -s /downloads/git "${BUILDCHROOT_DIR}/home/.git-downloads"
> +
> # Configure root filesystem
> sudo install -m 755 ${WORKDIR}/configscript.sh ${BUILDCHROOT_DIR}
> USER_ID=$(id -u)
>
--
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2021-08-12 10:31 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-12 9:23 [PATCH 0/3] Make adjust_git work in both worlds Jan Kiszka
2021-08-12 9:23 ` [PATCH 1/3] dpkg-base: Lock do_adjust_git against each other Jan Kiszka
2021-08-12 9:23 ` [PATCH 2/3] dpkg-base: Make mirror link relative Jan Kiszka
2021-08-12 9:23 ` [PATCH 3/3] Rework do_adjust_git to support inside and outside usage Jan Kiszka
2021-08-12 10:31 ` Jan Kiszka
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox