* [PATCH v3 1/4] dpkg-base: Lock do_adjust_git against each other
2021-08-26 9:22 [PATCH v3 0/4] Make adjust_git work in both worlds Jan Kiszka
@ 2021-08-26 9:22 ` Jan Kiszka
2021-08-26 9:22 ` [PATCH v3 2/4] dpkg-base: Make mirror link relative Jan Kiszka
` (3 subsequent siblings)
4 siblings, 0 replies; 8+ messages in thread
From: Jan Kiszka @ 2021-08-26 9:22 UTC (permalink / raw)
To: isar-users; +Cc: Uladzimir Bely, Henning Schild
From: Jan Kiszka <jan.kiszka@siemens.com>
This is needed since e47ada143469 because we now adjust the links to
mirrors in DL_DIR/git which may collide with other tasks doing this at
the same time.
Locking will also be needed soon for establishing the host-side link
to the download dir.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
meta/classes/dpkg-base.bbclass | 1 +
1 file changed, 1 insertion(+)
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index ec8fbc1..2cff1b3 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -55,6 +55,7 @@ python do_adjust_git() {
}
addtask adjust_git before do_dpkg_build
+do_adjust_git[lockfiles] += "${DL_DIR}/git/isar.lock"
inherit patch
addtask patch before do_adjust_git
--
2.31.1
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH v3 2/4] dpkg-base: Make mirror link relative
2021-08-26 9:22 [PATCH v3 0/4] Make adjust_git work in both worlds Jan Kiszka
2021-08-26 9:22 ` [PATCH v3 1/4] dpkg-base: Lock do_adjust_git against each other Jan Kiszka
@ 2021-08-26 9:22 ` Jan Kiszka
2021-08-26 9:22 ` [PATCH v3 3/4] Rework do_adjust_git to support inside and outside usage Jan Kiszka
` (2 subsequent siblings)
4 siblings, 0 replies; 8+ messages in thread
From: Jan Kiszka @ 2021-08-26 9:22 UTC (permalink / raw)
To: isar-users; +Cc: Uladzimir Bely, Henning Schild
From: Jan Kiszka <jan.kiszka@siemens.com>
This will allow to use it both from the outer builder context and inside
the buildchroot. No idea why bitbake doesn't do that in the first place.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
meta/classes/dpkg-base.bbclass | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index 2cff1b3..ca1dccf 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -28,8 +28,10 @@ python do_adjust_git() {
if os.path.islink(ud.localpath):
realpath = os.path.realpath(ud.localpath)
- if realpath.startswith(d.getVar("DL_DIR")):
- link = realpath.replace(d.getVar("DL_DIR"), '/downloads', 1)
+ filter_out = os.path.join(d.getVar("DL_DIR"), "git") + "/"
+ if realpath.startswith(filter_out):
+ # make the link relative
+ link = realpath.replace(filter_out, '', 1)
os.unlink(ud.localpath)
os.symlink(link, ud.localpath)
--
2.31.1
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH v3 3/4] Rework do_adjust_git to support inside and outside usage
2021-08-26 9:22 [PATCH v3 0/4] Make adjust_git work in both worlds Jan Kiszka
2021-08-26 9:22 ` [PATCH v3 1/4] dpkg-base: Lock do_adjust_git against each other Jan Kiszka
2021-08-26 9:22 ` [PATCH v3 2/4] dpkg-base: Make mirror link relative Jan Kiszka
@ 2021-08-26 9:22 ` Jan Kiszka
2021-08-26 9:22 ` [PATCH v3 4/4] Revert "dpkg: adjust task order to allow using "git" for patching" Jan Kiszka
2021-09-09 10:42 ` [PATCH v3 0/4] Make adjust_git work in both worlds Anton Mikanovich
4 siblings, 0 replies; 8+ messages in thread
From: Jan Kiszka @ 2021-08-26 9:22 UTC (permalink / raw)
To: isar-users; +Cc: Uladzimir Bely, Henning Schild
From: Jan Kiszka <jan.kiszka@siemens.com>
Rewrite the alternates entry to use a relative path outside of the
workdir. That linke can then be created independently of the outer build
environment as well as inside the buildchroot.
We use ${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/.git-downloads outside
and /home/.git-downloads inside the buildchroot.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
meta/classes/dpkg-base.bbclass | 11 ++++++++++-
meta/conf/bitbake.conf | 1 +
meta/recipes-devtools/buildchroot/buildchroot.inc | 2 ++
3 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index ca1dccf..9a18acb 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -19,6 +19,12 @@ python do_adjust_git() {
rootdir = d.getVar('WORKDIR', True)
+ git_link = os.path.join(d.getVar('GIT_DL_LINK_DIR'), '.git-downloads')
+ git_dl = os.path.join(d.getVar("DL_DIR"), "git")
+
+ if not os.path.exists(git_link) or os.path.realpath(git_link) != git_dl:
+ os.symlink(git_dl, git_link)
+
for src_uri in (d.getVar("SRC_URI", True) or "").split():
try:
fetcher = bb.fetch2.Fetch([src_uri], d)
@@ -44,11 +50,14 @@ python do_adjust_git() {
destsuffix = ud.parm.get("destsuffix", def_destsuffix)
destdir = ud.destdir = os.path.join(rootdir, destsuffix)
+ git_link_rel = os.path.relpath(git_link,
+ os.path.join(destdir, ".git/objects"))
+
alternates = os.path.join(destdir, ".git/objects/info/alternates")
if os.path.exists(alternates):
cmd = ["sed", "-i", alternates, "-e",
- "s|{}|/downloads|".format(d.getVar("DL_DIR"))]
+ "s|{}|{}|".format(git_dl, git_link_rel)]
bb.note(' '.join(cmd))
if subprocess.call(cmd) != 0:
bb.fatal("git alternates adjustment failed")
diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
index 9859456..7f5901d 100644
--- a/meta/conf/bitbake.conf
+++ b/meta/conf/bitbake.conf
@@ -48,6 +48,7 @@ GITPKGV = "${@bb.fetch2.get_srcrev(d, 'gitpkgv_revision')}"
# isar specific config
WORKDIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}/${PN}/${PV}-${PR}"
+GIT_DL_LINK_DIR = "${TMPDIR}/work/${DISTRO}-${DISTRO_ARCH}"
DEPLOY_DIR_BOOTSTRAP = "${DEPLOY_DIR}/bootstrap"
DEPLOY_DIR_BUILDCHROOT = "${DEPLOY_DIR}/buildchroot"
DEPLOY_DIR_SDKCHROOT = "${DEPLOY_DIR}/sdkchroot"
diff --git a/meta/recipes-devtools/buildchroot/buildchroot.inc b/meta/recipes-devtools/buildchroot/buildchroot.inc
index 0c1fb7f..52a5f1c 100644
--- a/meta/recipes-devtools/buildchroot/buildchroot.inc
+++ b/meta/recipes-devtools/buildchroot/buildchroot.inc
@@ -67,6 +67,8 @@ buildchroot_install_files() {
sudo install -m 755 ${WORKDIR}/common.sh ${BUILDCHROOT_DIR}/isar/
sudo install -m 755 ${WORKDIR}/deps.sh ${BUILDCHROOT_DIR}/isar/
+ sudo ln -s /downloads/git "${BUILDCHROOT_DIR}/home/.git-downloads"
+
# Configure root filesystem
sudo install -m 755 ${WORKDIR}/configscript.sh ${BUILDCHROOT_DIR}
USER_ID=$(id -u)
--
2.31.1
^ permalink raw reply [flat|nested] 8+ messages in thread* [PATCH v3 4/4] Revert "dpkg: adjust task order to allow using "git" for patching"
2021-08-26 9:22 [PATCH v3 0/4] Make adjust_git work in both worlds Jan Kiszka
` (2 preceding siblings ...)
2021-08-26 9:22 ` [PATCH v3 3/4] Rework do_adjust_git to support inside and outside usage Jan Kiszka
@ 2021-08-26 9:22 ` Jan Kiszka
2021-09-09 10:42 ` [PATCH v3 0/4] Make adjust_git work in both worlds Anton Mikanovich
4 siblings, 0 replies; 8+ messages in thread
From: Jan Kiszka @ 2021-08-26 9:22 UTC (permalink / raw)
To: isar-users; +Cc: Uladzimir Bely, Henning Schild
From: Jan Kiszka <jan.kiszka@siemens.com>
This reverts commit b4c33ba69ed89849ecf32d8731aa97ecf88226f5.
No longer needed now that adjust_git stopped breaking the outer
environment. Moreover, this was broken /wrt devshell anyway.
Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
meta/classes/dpkg-base.bbclass | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index 9a18acb..8286168 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -65,11 +65,11 @@ python do_adjust_git() {
bb.fatal(str(e))
}
-addtask adjust_git before do_dpkg_build
+addtask adjust_git after do_unpack before do_patch
do_adjust_git[lockfiles] += "${DL_DIR}/git/isar.lock"
inherit patch
-addtask patch before do_adjust_git
+addtask patch after do_adjust_git before do_dpkg_build
SRC_APT ?= ""
--
2.31.1
^ permalink raw reply [flat|nested] 8+ messages in thread* Re: [PATCH v3 0/4] Make adjust_git work in both worlds
2021-08-26 9:22 [PATCH v3 0/4] Make adjust_git work in both worlds Jan Kiszka
` (3 preceding siblings ...)
2021-08-26 9:22 ` [PATCH v3 4/4] Revert "dpkg: adjust task order to allow using "git" for patching" Jan Kiszka
@ 2021-09-09 10:42 ` Anton Mikanovich
4 siblings, 0 replies; 8+ messages in thread
From: Anton Mikanovich @ 2021-09-09 10:42 UTC (permalink / raw)
To: Jan Kiszka, isar-users; +Cc: Uladzimir Bely, Henning Schild
26.08.2021 12:22, Jan Kiszka wrote:
> Finally allow the adjusted git repos work both inside and outside the
> buildchroots. Also fixes the race introduced by patching links in
> DL_DIR.
>
> Changes in v3:
> - use /git for lock
> - fix typos in commit logs
>
> Changes in v2:
> - build relative alternates path so that destsuffix paths are respected
> - fix task dependencies so that devshell gets adjusted git again
>
> Jan
>
> Jan Kiszka (4):
> dpkg-base: Lock do_adjust_git against each other
> dpkg-base: Make mirror link relative
> Rework do_adjust_git to support inside and outside usage
> Revert "dpkg: adjust task order to allow using "git" for patching"
>
> meta/classes/dpkg-base.bbclass | 22 ++++++++++++++-----
> meta/conf/bitbake.conf | 1 +
> .../buildchroot/buildchroot.inc | 2 ++
> 3 files changed, 20 insertions(+), 5 deletions(-)
>
Applied to next, thanks.
--
Anton Mikanovich
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov
^ permalink raw reply [flat|nested] 8+ messages in thread