* Re: [PATCH] dpkg-base: cope with race around check/creation of .git-downloads symlink
2022-02-02 7:05 ` Jan Kiszka
@ 2022-02-04 8:34 ` Cedric Hombourger
2022-02-04 10:50 ` [[PATCH v2]] dpkg-base: resolve DL_DIR in do_adjust_git Cedric Hombourger
2022-02-04 10:54 ` [PATCH v3] " Cedric Hombourger
2 siblings, 0 replies; 13+ messages in thread
From: Cedric Hombourger @ 2022-02-04 8:34 UTC (permalink / raw)
To: Jan Kiszka, isar-users
On Wed, 2022-02-02 at 08:05 +0100, Jan Kiszka wrote:
> On 01.02.22 14:41, Cedric Hombourger wrote:
> > There is a race condition between the check for the .git-downloads
> > symbolic
> > link existing and its creation. Ignore the FileExistsError
> > exception when
> > hitting this race.
> >
> > Signed-off-by: Cedric Hombourger <Cedric_Hombourger@mentor.com>
> > ---
> > meta/classes/dpkg-base.bbclass | 5 ++++-
> > 1 file changed, 4 insertions(+), 1 deletion(-)
> >
> > diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-
> > base.bbclass
> > index 2add0b2..3aa52b1 100644
> > --- a/meta/classes/dpkg-base.bbclass
> > +++ b/meta/classes/dpkg-base.bbclass
> > @@ -23,7 +23,10 @@ python do_adjust_git() {
> > git_dl = os.path.join(d.getVar("DL_DIR"), "git")
> >
> > if not os.path.exists(git_link) or os.path.realpath(git_link)
> > != git_dl:
> > - os.symlink(git_dl, git_link)
> > + try:
> > + os.symlink(git_dl, git_link)
> > + except FileExistsError:
> > + pass
> >
> > for src_uri in (d.getVar("SRC_URI", True) or "").split():
> > try:
>
> do_adjust_git is run under global ${DL_DIR}/git/isar.lock - how do
> you
> trigger this race then?
good catch. I had not noticed this lock. I have been getting this
exception while building on a machine with many CPUs (32 IIRC). I have
no idea how this could happen with this lock held. Going to debug
further. Thanks for the pointer
>
> Jan
>
^ permalink raw reply [flat|nested] 13+ messages in thread* [[PATCH v2]] dpkg-base: resolve DL_DIR in do_adjust_git
2022-02-02 7:05 ` Jan Kiszka
2022-02-04 8:34 ` Cedric Hombourger
@ 2022-02-04 10:50 ` Cedric Hombourger
2022-02-04 10:54 ` [PATCH v3] " Cedric Hombourger
2 siblings, 0 replies; 13+ messages in thread
From: Cedric Hombourger @ 2022-02-04 10:50 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger
From: Cedric Hombourger <cedric.hombourger@siemens.com>
git_link is resolved using os.path.realpath() but git_dl is not.
If DL_DIR points to a symbolic link, the comparison will always
fail and do_adjust_git() will attempt to re-create the symbolic
link. Resolve DL_DIR for a comparison between resolved paths.
In the event where paths do differ, the symbolic link needs to
be deleted first.
Disposition: Submit upstream
Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
---
meta/classes/dpkg-base.bbclass | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index 2add0b2..258e040 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -20,9 +20,13 @@ python do_adjust_git() {
rootdir = d.getVar('WORKDIR', True)
git_link = os.path.join(d.getVar('GIT_DL_LINK_DIR'), '.git-downloads')
- git_dl = os.path.join(d.getVar("DL_DIR"), "git")
+ dl_dir = os.path.realpath(d.getVar("DL_DIR"))
+ git_dl = os.path.join(dl_dir, "git")
- if not os.path.exists(git_link) or os.path.realpath(git_link) != git_dl:
+ if os.path.exists(git_link) and os.path.realpath(git_link) != git_dl:
+ os.unlink(git_link)
+
+ if not os.path.exists(git_link):
os.symlink(git_dl, git_link)
for src_uri in (d.getVar("SRC_URI", True) or "").split():
@@ -34,7 +38,7 @@ python do_adjust_git() {
if os.path.islink(ud.localpath):
realpath = os.path.realpath(ud.localpath)
- filter_out = os.path.join(d.getVar("DL_DIR"), "git") + "/"
+ filter_out = git_dl + "/"
if realpath.startswith(filter_out):
# make the link relative
link = realpath.replace(filter_out, '', 1)
--
2.30.2
^ permalink raw reply [flat|nested] 13+ messages in thread* [PATCH v3] dpkg-base: resolve DL_DIR in do_adjust_git
2022-02-02 7:05 ` Jan Kiszka
2022-02-04 8:34 ` Cedric Hombourger
2022-02-04 10:50 ` [[PATCH v2]] dpkg-base: resolve DL_DIR in do_adjust_git Cedric Hombourger
@ 2022-02-04 10:54 ` Cedric Hombourger
2022-02-04 11:04 ` Jan Kiszka
2022-02-04 12:30 ` [PATCH v3] " Henning Schild
2 siblings, 2 replies; 13+ messages in thread
From: Cedric Hombourger @ 2022-02-04 10:54 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger
From: Cedric Hombourger <cedric.hombourger@siemens.com>
git_link is resolved using os.path.realpath() but git_dl is not.
If DL_DIR points to a symbolic link, the comparison will always
fail and do_adjust_git() will attempt to re-create the symbolic
link. Resolve DL_DIR for a comparison between resolved paths.
In the event where paths do differ, the symbolic link needs to
be deleted first.
Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
---
meta/classes/dpkg-base.bbclass | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index 2add0b2..258e040 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -20,9 +20,13 @@ python do_adjust_git() {
rootdir = d.getVar('WORKDIR', True)
git_link = os.path.join(d.getVar('GIT_DL_LINK_DIR'), '.git-downloads')
- git_dl = os.path.join(d.getVar("DL_DIR"), "git")
+ dl_dir = os.path.realpath(d.getVar("DL_DIR"))
+ git_dl = os.path.join(dl_dir, "git")
- if not os.path.exists(git_link) or os.path.realpath(git_link) != git_dl:
+ if os.path.exists(git_link) and os.path.realpath(git_link) != git_dl:
+ os.unlink(git_link)
+
+ if not os.path.exists(git_link):
os.symlink(git_dl, git_link)
for src_uri in (d.getVar("SRC_URI", True) or "").split():
@@ -34,7 +38,7 @@ python do_adjust_git() {
if os.path.islink(ud.localpath):
realpath = os.path.realpath(ud.localpath)
- filter_out = os.path.join(d.getVar("DL_DIR"), "git") + "/"
+ filter_out = git_dl + "/"
if realpath.startswith(filter_out):
# make the link relative
link = realpath.replace(filter_out, '', 1)
--
2.30.2
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH v3] dpkg-base: resolve DL_DIR in do_adjust_git
2022-02-04 10:54 ` [PATCH v3] " Cedric Hombourger
@ 2022-02-04 11:04 ` Jan Kiszka
2022-02-04 12:45 ` Cedric Hombourger
2022-02-04 12:30 ` [PATCH v3] " Henning Schild
1 sibling, 1 reply; 13+ messages in thread
From: Jan Kiszka @ 2022-02-04 11:04 UTC (permalink / raw)
To: Cedric Hombourger, isar-users; +Cc: Cedric Hombourger
On 04.02.22 11:54, Cedric Hombourger wrote:
> From: Cedric Hombourger <cedric.hombourger@siemens.com>
>
> git_link is resolved using os.path.realpath() but git_dl is not.
> If DL_DIR points to a symbolic link, the comparison will always
> fail and do_adjust_git() will attempt to re-create the symbolic
> link. Resolve DL_DIR for a comparison between resolved paths.
> In the event where paths do differ, the symbolic link needs to
> be deleted first.
>
> Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
> ---
> meta/classes/dpkg-base.bbclass | 10 +++++++---
> 1 file changed, 7 insertions(+), 3 deletions(-)
>
> diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
> index 2add0b2..258e040 100644
> --- a/meta/classes/dpkg-base.bbclass
> +++ b/meta/classes/dpkg-base.bbclass
> @@ -20,9 +20,13 @@ python do_adjust_git() {
> rootdir = d.getVar('WORKDIR', True)
>
> git_link = os.path.join(d.getVar('GIT_DL_LINK_DIR'), '.git-downloads')
> - git_dl = os.path.join(d.getVar("DL_DIR"), "git")
> + dl_dir = os.path.realpath(d.getVar("DL_DIR"))
> + git_dl = os.path.join(dl_dir, "git")
>
> - if not os.path.exists(git_link) or os.path.realpath(git_link) != git_dl:
> + if os.path.exists(git_link) and os.path.realpath(git_link) != git_dl:
> + os.unlink(git_link)
> +
> + if not os.path.exists(git_link):
> os.symlink(git_dl, git_link)
>
> for src_uri in (d.getVar("SRC_URI", True) or "").split():
> @@ -34,7 +38,7 @@ python do_adjust_git() {
>
> if os.path.islink(ud.localpath):
> realpath = os.path.realpath(ud.localpath)
> - filter_out = os.path.join(d.getVar("DL_DIR"), "git") + "/"
> + filter_out = git_dl + "/"
> if realpath.startswith(filter_out):
> # make the link relative
> link = realpath.replace(filter_out, '', 1)
Looks good and was surely an untested case so far. It's actually two
fixes in one (os.symlink != ln -sf...)
Does this explain your "race condition" from before?
Jan
--
Siemens AG, Technology
Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH v3] dpkg-base: resolve DL_DIR in do_adjust_git
2022-02-04 11:04 ` Jan Kiszka
@ 2022-02-04 12:45 ` Cedric Hombourger
2022-02-09 8:12 ` [PATCH v4 0/1] " Cedric Hombourger
0 siblings, 1 reply; 13+ messages in thread
From: Cedric Hombourger @ 2022-02-04 12:45 UTC (permalink / raw)
To: Jan Kiszka, isar-users; +Cc: Cedric Hombourger
On Fri, 2022-02-04 at 12:04 +0100, Jan Kiszka wrote:
> On 04.02.22 11:54, Cedric Hombourger wrote:
> > From: Cedric Hombourger <cedric.hombourger@siemens.com>
> >
> > git_link is resolved using os.path.realpath() but git_dl is not.
> > If DL_DIR points to a symbolic link, the comparison will always
> > fail and do_adjust_git() will attempt to re-create the symbolic
> > link. Resolve DL_DIR for a comparison between resolved paths.
> > In the event where paths do differ, the symbolic link needs to
> > be deleted first.
> >
> > Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
> > ---
> > meta/classes/dpkg-base.bbclass | 10 +++++++---
> > 1 file changed, 7 insertions(+), 3 deletions(-)
> >
> > diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-
> > base.bbclass
> > index 2add0b2..258e040 100644
> > --- a/meta/classes/dpkg-base.bbclass
> > +++ b/meta/classes/dpkg-base.bbclass
> > @@ -20,9 +20,13 @@ python do_adjust_git() {
> > rootdir = d.getVar('WORKDIR', True)
> >
> > git_link = os.path.join(d.getVar('GIT_DL_LINK_DIR'), '.git-
> > downloads')
> > - git_dl = os.path.join(d.getVar("DL_DIR"), "git")
> > + dl_dir = os.path.realpath(d.getVar("DL_DIR"))
> > + git_dl = os.path.join(dl_dir, "git")
> >
> > - if not os.path.exists(git_link) or os.path.realpath(git_link)
> > != git_dl:
> > + if os.path.exists(git_link) and os.path.realpath(git_link) !=
> > git_dl:
> > + os.unlink(git_link)
> > +
> > + if not os.path.exists(git_link):
> > os.symlink(git_dl, git_link)
> >
> > for src_uri in (d.getVar("SRC_URI", True) or "").split():
> > @@ -34,7 +38,7 @@ python do_adjust_git() {
> >
> > if os.path.islink(ud.localpath):
> > realpath = os.path.realpath(ud.localpath)
> > - filter_out = os.path.join(d.getVar("DL_DIR"),
> > "git") + "/"
> > + filter_out = git_dl + "/"
> > if realpath.startswith(filter_out):
> > # make the link relative
> > link = realpath.replace(filter_out, '', 1)
>
> Looks good and was surely an untested case so far. It's actually two
> fixes in one (os.symlink != ln -sf...)
>
> Does this explain your "race condition" from before?
Yes it certainly did. I had misinterpreted the symptoms / root-cause
(thanks again for pointing me to the lock around do_adjust_git)
>
> Jan
>
^ permalink raw reply [flat|nested] 13+ messages in thread* [PATCH v4 0/1] dpkg-base: resolve DL_DIR in do_adjust_git
2022-02-04 12:45 ` Cedric Hombourger
@ 2022-02-09 8:12 ` Cedric Hombourger
2022-02-09 8:12 ` [PATCH v4 1/1] " Cedric Hombourger
2022-02-21 9:25 ` [PATCH v4 0/1] " Anton Mikanovich
0 siblings, 2 replies; 13+ messages in thread
From: Cedric Hombourger @ 2022-02-09 8:12 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger
Changes since v3:
The path placed in .git/objects/info/alternates shall be relative,
we should resolve git symbolic links only when comparing them
v3 changes:
fix subject line
v2 changes:
found actual root cause. instead of catching FileExistsError, we
should instead resolve symbolic links and compare them. In the
event when they do not match, the previous symbolic link should
be re-created.
Cedric Hombourger (1):
dpkg-base: resolve DL_DIR in do_adjust_git
meta/classes/dpkg-base.bbclass | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
--
2.30.2
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH v4 1/1] dpkg-base: resolve DL_DIR in do_adjust_git
2022-02-09 8:12 ` [PATCH v4 0/1] " Cedric Hombourger
@ 2022-02-09 8:12 ` Cedric Hombourger
2022-02-21 9:25 ` [PATCH v4 0/1] " Anton Mikanovich
1 sibling, 0 replies; 13+ messages in thread
From: Cedric Hombourger @ 2022-02-09 8:12 UTC (permalink / raw)
To: isar-users; +Cc: Cedric Hombourger
From: Cedric Hombourger <cedric.hombourger@siemens.com>
git_link is resolved using os.path.realpath() but git_dl is not.
If DL_DIR points to a symbolic link, the comparison will always
fail and do_adjust_git() will attempt to re-create the symbolic
link. Resolve DL_DIR for a comparison between resolved paths.
In the event where paths do differ, the symbolic link needs to
be deleted first.
Disposition: Submit upstream
Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
---
meta/classes/dpkg-base.bbclass | 10 +++++++---
1 file changed, 7 insertions(+), 3 deletions(-)
diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index 2add0b2..4d496fc 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -20,9 +20,13 @@ python do_adjust_git() {
rootdir = d.getVar('WORKDIR', True)
git_link = os.path.join(d.getVar('GIT_DL_LINK_DIR'), '.git-downloads')
- git_dl = os.path.join(d.getVar("DL_DIR"), "git")
+ dl_dir = d.getVar("DL_DIR")
+ git_dl = os.path.join(dl_dir, "git")
- if not os.path.exists(git_link) or os.path.realpath(git_link) != git_dl:
+ if os.path.exists(git_link) and os.path.realpath(git_link) != os.path.realpath(git_dl):
+ os.unlink(git_link)
+
+ if not os.path.exists(git_link):
os.symlink(git_dl, git_link)
for src_uri in (d.getVar("SRC_URI", True) or "").split():
@@ -34,7 +38,7 @@ python do_adjust_git() {
if os.path.islink(ud.localpath):
realpath = os.path.realpath(ud.localpath)
- filter_out = os.path.join(d.getVar("DL_DIR"), "git") + "/"
+ filter_out = git_dl + "/"
if realpath.startswith(filter_out):
# make the link relative
link = realpath.replace(filter_out, '', 1)
--
2.30.2
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH v4 0/1] dpkg-base: resolve DL_DIR in do_adjust_git
2022-02-09 8:12 ` [PATCH v4 0/1] " Cedric Hombourger
2022-02-09 8:12 ` [PATCH v4 1/1] " Cedric Hombourger
@ 2022-02-21 9:25 ` Anton Mikanovich
1 sibling, 0 replies; 13+ messages in thread
From: Anton Mikanovich @ 2022-02-21 9:25 UTC (permalink / raw)
To: Cedric Hombourger, isar-users
9.02.22 11:12, Cedric Hombourger wrote:
> Changes since v3:
> The path placed in .git/objects/info/alternates shall be relative,
> we should resolve git symbolic links only when comparing them
>
> v3 changes:
> fix subject line
>
> v2 changes:
> found actual root cause. instead of catching FileExistsError, we
> should instead resolve symbolic links and compare them. In the
> event when they do not match, the previous symbolic link should
> be re-created.
>
> Cedric Hombourger (1):
> dpkg-base: resolve DL_DIR in do_adjust_git
>
> meta/classes/dpkg-base.bbclass | 10 +++++++---
> 1 file changed, 7 insertions(+), 3 deletions(-)
>
Applied to next, thanks.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH v3] dpkg-base: resolve DL_DIR in do_adjust_git
2022-02-04 10:54 ` [PATCH v3] " Cedric Hombourger
2022-02-04 11:04 ` Jan Kiszka
@ 2022-02-04 12:30 ` Henning Schild
2022-02-04 12:43 ` Jan Kiszka
2023-02-16 12:57 ` Henning Schild
1 sibling, 2 replies; 13+ messages in thread
From: Henning Schild @ 2022-02-04 12:30 UTC (permalink / raw)
To: Cedric Hombourger; +Cc: isar-users, Cedric Hombourger
Working with sstate i found a very similiar thing but in a very
different place.
https://github.com/henning-schild-work/isar/commit/d7eff82427fbe8e4f5ef50268804257fc9c143a0
Makes me wonder who is responsible for creating the link, or if every
stakeholder is just hammering around.
In fact i still vote for dropping adjust git and using "no shared"
cloning.
Henning
Am Fri, 4 Feb 2022 11:54:18 +0100
schrieb Cedric Hombourger <Cedric_Hombourger@mentor.com>:
> From: Cedric Hombourger <cedric.hombourger@siemens.com>
>
> git_link is resolved using os.path.realpath() but git_dl is not.
> If DL_DIR points to a symbolic link, the comparison will always
> fail and do_adjust_git() will attempt to re-create the symbolic
> link. Resolve DL_DIR for a comparison between resolved paths.
> In the event where paths do differ, the symbolic link needs to
> be deleted first.
>
> Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
> ---
> meta/classes/dpkg-base.bbclass | 10 +++++++---
> 1 file changed, 7 insertions(+), 3 deletions(-)
>
> diff --git a/meta/classes/dpkg-base.bbclass
> b/meta/classes/dpkg-base.bbclass index 2add0b2..258e040 100644
> --- a/meta/classes/dpkg-base.bbclass
> +++ b/meta/classes/dpkg-base.bbclass
> @@ -20,9 +20,13 @@ python do_adjust_git() {
> rootdir = d.getVar('WORKDIR', True)
>
> git_link = os.path.join(d.getVar('GIT_DL_LINK_DIR'),
> '.git-downloads')
> - git_dl = os.path.join(d.getVar("DL_DIR"), "git")
> + dl_dir = os.path.realpath(d.getVar("DL_DIR"))
> + git_dl = os.path.join(dl_dir, "git")
>
> - if not os.path.exists(git_link) or os.path.realpath(git_link) !=
> git_dl:
> + if os.path.exists(git_link) and os.path.realpath(git_link) !=
> git_dl:
> + os.unlink(git_link)
> +
> + if not os.path.exists(git_link):
> os.symlink(git_dl, git_link)
>
> for src_uri in (d.getVar("SRC_URI", True) or "").split():
> @@ -34,7 +38,7 @@ python do_adjust_git() {
>
> if os.path.islink(ud.localpath):
> realpath = os.path.realpath(ud.localpath)
> - filter_out = os.path.join(d.getVar("DL_DIR"), "git")
> + "/"
> + filter_out = git_dl + "/"
> if realpath.startswith(filter_out):
> # make the link relative
> link = realpath.replace(filter_out, '', 1)
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH v3] dpkg-base: resolve DL_DIR in do_adjust_git
2022-02-04 12:30 ` [PATCH v3] " Henning Schild
@ 2022-02-04 12:43 ` Jan Kiszka
2023-02-16 12:57 ` Henning Schild
1 sibling, 0 replies; 13+ messages in thread
From: Jan Kiszka @ 2022-02-04 12:43 UTC (permalink / raw)
To: Henning Schild, Cedric Hombourger; +Cc: isar-users, Cedric Hombourger
On 04.02.22 13:30, Henning Schild wrote:
> Working with sstate i found a very similiar thing but in a very
> different place.
>
> https://github.com/henning-schild-work/isar/commit/d7eff82427fbe8e4f5ef50268804257fc9c143a0
That is a re-execution issue of a task, not a race.
>
> Makes me wonder who is responsible for creating the link, or if every
> stakeholder is just hammering around.
The first one coming along - one reason why it's under a lock.
Jan
>
> In fact i still vote for dropping adjust git and using "no shared"
> cloning.
>
> Henning
>
> Am Fri, 4 Feb 2022 11:54:18 +0100
> schrieb Cedric Hombourger <Cedric_Hombourger@mentor.com>:
>
>> From: Cedric Hombourger <cedric.hombourger@siemens.com>
>>
>> git_link is resolved using os.path.realpath() but git_dl is not.
>> If DL_DIR points to a symbolic link, the comparison will always
>> fail and do_adjust_git() will attempt to re-create the symbolic
>> link. Resolve DL_DIR for a comparison between resolved paths.
>> In the event where paths do differ, the symbolic link needs to
>> be deleted first.
>>
>> Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
>> ---
>> meta/classes/dpkg-base.bbclass | 10 +++++++---
>> 1 file changed, 7 insertions(+), 3 deletions(-)
>>
>> diff --git a/meta/classes/dpkg-base.bbclass
>> b/meta/classes/dpkg-base.bbclass index 2add0b2..258e040 100644
>> --- a/meta/classes/dpkg-base.bbclass
>> +++ b/meta/classes/dpkg-base.bbclass
>> @@ -20,9 +20,13 @@ python do_adjust_git() {
>> rootdir = d.getVar('WORKDIR', True)
>>
>> git_link = os.path.join(d.getVar('GIT_DL_LINK_DIR'),
>> '.git-downloads')
>> - git_dl = os.path.join(d.getVar("DL_DIR"), "git")
>> + dl_dir = os.path.realpath(d.getVar("DL_DIR"))
>> + git_dl = os.path.join(dl_dir, "git")
>>
>> - if not os.path.exists(git_link) or os.path.realpath(git_link) !=
>> git_dl:
>> + if os.path.exists(git_link) and os.path.realpath(git_link) !=
>> git_dl:
>> + os.unlink(git_link)
>> +
>> + if not os.path.exists(git_link):
>> os.symlink(git_dl, git_link)
>>
>> for src_uri in (d.getVar("SRC_URI", True) or "").split():
>> @@ -34,7 +38,7 @@ python do_adjust_git() {
>>
>> if os.path.islink(ud.localpath):
>> realpath = os.path.realpath(ud.localpath)
>> - filter_out = os.path.join(d.getVar("DL_DIR"), "git")
>> + "/"
>> + filter_out = git_dl + "/"
>> if realpath.startswith(filter_out):
>> # make the link relative
>> link = realpath.replace(filter_out, '', 1)
>
--
Siemens AG, Technology
Competence Center Embedded Linux
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH v3] dpkg-base: resolve DL_DIR in do_adjust_git
2022-02-04 12:30 ` [PATCH v3] " Henning Schild
2022-02-04 12:43 ` Jan Kiszka
@ 2023-02-16 12:57 ` Henning Schild
1 sibling, 0 replies; 13+ messages in thread
From: Henning Schild @ 2023-02-16 12:57 UTC (permalink / raw)
To: Jan Kiszka; +Cc: Cedric Hombourger, isar-users, Cedric Hombourger
Am Fri, 4 Feb 2022 13:30:36 +0100
schrieb Henning Schild <henning.schild@siemens.com>:
> Working with sstate i found a very similiar thing but in a very
> different place.
>
> https://github.com/henning-schild-work/isar/commit/d7eff82427fbe8e4f5ef50268804257fc9c143a0
>
> Makes me wonder who is responsible for creating the link, or if every
> stakeholder is just hammering around.
>
> In fact i still vote for dropping adjust git and using "no shared"
> cloning.
Yesterday i wanted to backport a patch into a component that already
gets patched with isar and git as patchtool. And since that patch did
not apply i needed to resolve merge conflicts.
Leaving git repos in unusable state on the host is just a massive pain
every once in a while. I do not want a devshell in a container, which
takes me miles away from where i want to be for a simple "git
cherry-pick" because an "git am" did not work.
I again ask to switch to "not shared" cloning, or to re-adjust when
coming back out of the chroots.
Anyone on my side? Last time i proposed patches i was overvoted, but
maybe the use-case or pain was not clear.
Henning
> Henning
>
> Am Fri, 4 Feb 2022 11:54:18 +0100
> schrieb Cedric Hombourger <Cedric_Hombourger@mentor.com>:
>
> > From: Cedric Hombourger <cedric.hombourger@siemens.com>
> >
> > git_link is resolved using os.path.realpath() but git_dl is not.
> > If DL_DIR points to a symbolic link, the comparison will always
> > fail and do_adjust_git() will attempt to re-create the symbolic
> > link. Resolve DL_DIR for a comparison between resolved paths.
> > In the event where paths do differ, the symbolic link needs to
> > be deleted first.
> >
> > Signed-off-by: Cedric Hombourger <cedric.hombourger@siemens.com>
> > ---
> > meta/classes/dpkg-base.bbclass | 10 +++++++---
> > 1 file changed, 7 insertions(+), 3 deletions(-)
> >
> > diff --git a/meta/classes/dpkg-base.bbclass
> > b/meta/classes/dpkg-base.bbclass index 2add0b2..258e040 100644
> > --- a/meta/classes/dpkg-base.bbclass
> > +++ b/meta/classes/dpkg-base.bbclass
> > @@ -20,9 +20,13 @@ python do_adjust_git() {
> > rootdir = d.getVar('WORKDIR', True)
> >
> > git_link = os.path.join(d.getVar('GIT_DL_LINK_DIR'),
> > '.git-downloads')
> > - git_dl = os.path.join(d.getVar("DL_DIR"), "git")
> > + dl_dir = os.path.realpath(d.getVar("DL_DIR"))
> > + git_dl = os.path.join(dl_dir, "git")
> >
> > - if not os.path.exists(git_link) or os.path.realpath(git_link)
> > != git_dl:
> > + if os.path.exists(git_link) and os.path.realpath(git_link) !=
> > git_dl:
> > + os.unlink(git_link)
> > +
> > + if not os.path.exists(git_link):
> > os.symlink(git_dl, git_link)
> >
> > for src_uri in (d.getVar("SRC_URI", True) or "").split():
> > @@ -34,7 +38,7 @@ python do_adjust_git() {
> >
> > if os.path.islink(ud.localpath):
> > realpath = os.path.realpath(ud.localpath)
> > - filter_out = os.path.join(d.getVar("DL_DIR"),
> > "git")
> > + "/"
> > + filter_out = git_dl + "/"
> > if realpath.startswith(filter_out):
> > # make the link relative
> > link = realpath.replace(filter_out, '', 1)
>
^ permalink raw reply [flat|nested] 13+ messages in thread