[RFC 0/2] Fixes for dpkg-gdb packages rebuild

[RFC 0/2] Fixes for dpkg-gdb packages rebuild

[Uladzimir Bely]

Second isar build can cause some packages rebuild due to configuration
changes and some of thems (like cowsay) fail due to broken git tree
and/or already being applied.

Uladzimir Bely (2):
  dpkg-base: Add new git alternates record instead of replace old one
  dpkg-gdb: Reset git to SRCREV revision before patching

 meta/classes/dpkg-base.bbclass |  2 +-
 meta/classes/dpkg-gbp.bbclass  | 10 ++++++++++
 2 files changed, 11 insertions(+), 1 deletion(-)

-- 
2.20.1

[RFC 1/2] dpkg-base: Add new git alternates record instead of replace old one

[Uladzimir Bely]

Task adjust_git modifies git alternates by replacing DL_DIR to /downloads.
New path is used inside buildchroot, so that working git repo
becomes broken outside buildchroot.

This causes problems with rebuilding such packages as cowsay if something
has changed in bitbake environment since previous build:

ERROR:
  mc:qemuarm64-buster:cowsay-git-r0 do_patch: Command Error: 'sh -c 'git
  --work-tree=/path/build/tmp/work/debian-buster-arm64/cowsay/git-r0/git
  reset --hard HEAD'' exited with 0

The solution is to add new path to git alternates so that original path
will work outside buildchroot, and new one will work in buildchroot.

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 meta/classes/dpkg-base.bbclass | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
index ec8fbc1..4b9b782 100644
--- a/meta/classes/dpkg-base.bbclass
+++ b/meta/classes/dpkg-base.bbclass
@@ -46,7 +46,7 @@ python do_adjust_git() {
 
             if os.path.exists(alternates):
                 cmd = ["sed", "-i", alternates, "-e",
-                       "s|{}|/downloads|".format(d.getVar("DL_DIR"))]
+                       "s|\({}\(.*\)\)|\\1\\n/downloads\\2|".format(d.getVar("DL_DIR"))]
                 bb.note(' '.join(cmd))
                 if subprocess.call(cmd) != 0:
                     bb.fatal("git alternates adjustment failed")
-- 
2.20.1

Re: [RFC 1/2] dpkg-base: Add new git alternates record instead of replace old one

[Jan Kiszka]

On 12.08.21 10:10, Uladzimir Bely wrote:
> Task adjust_git modifies git alternates by replacing DL_DIR to /downloads.
> New path is used inside buildchroot, so that working git repo
> becomes broken outside buildchroot.
> 
> This causes problems with rebuilding such packages as cowsay if something
> has changed in bitbake environment since previous build:
> 
> ERROR:
>   mc:qemuarm64-buster:cowsay-git-r0 do_patch: Command Error: 'sh -c 'git
>   --work-tree=/path/build/tmp/work/debian-buster-arm64/cowsay/git-r0/git
>   reset --hard HEAD'' exited with 0
> 
> The solution is to add new path to git alternates so that original path
> will work outside buildchroot, and new one will work in buildchroot.
> 
> Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
> ---
>  meta/classes/dpkg-base.bbclass | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/meta/classes/dpkg-base.bbclass b/meta/classes/dpkg-base.bbclass
> index ec8fbc1..4b9b782 100644
> --- a/meta/classes/dpkg-base.bbclass
> +++ b/meta/classes/dpkg-base.bbclass
> @@ -46,7 +46,7 @@ python do_adjust_git() {
>  
>              if os.path.exists(alternates):
>                  cmd = ["sed", "-i", alternates, "-e",
> -                       "s|{}|/downloads|".format(d.getVar("DL_DIR"))]
> +                       "s|\({}\(.*\)\)|\\1\\n/downloads\\2|".format(d.getVar("DL_DIR"))]
>                  bb.note(' '.join(cmd))
>                  if subprocess.call(cmd) != 0:
>                      bb.fatal("git alternates adjustment failed")
> 

This doesn'tt work, already tried: git fails if just one record is not
working.

Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

Re: [RFC 1/2] dpkg-base: Add new git alternates record instead of replace old one

[Uladzimir Bely]

[-- Attachment #1.1: Type: text/plain, Size: 1982 bytes --]

Yes, git still shows the message a currently invalid alternate (one of two 
records), but it's able to work further (reset --hard, etc).

But I can agree that this way could be unacceptable due this error message.

четверг, 12 августа 2021 г. в 12:00:58 UTC+3, Jan Kiszka: 

> On 12.08.21 10:10, Uladzimir Bely wrote:
> > Task adjust_git modifies git alternates by replacing DL_DIR to 
> /downloads.
> > New path is used inside buildchroot, so that working git repo
> > becomes broken outside buildchroot.
> > 
> > This causes problems with rebuilding such packages as cowsay if something
> > has changed in bitbake environment since previous build:
> > 
> > ERROR:
> > mc:qemuarm64-buster:cowsay-git-r0 do_patch: Command Error: 'sh -c 'git
> > --work-tree=/path/build/tmp/work/debian-buster-arm64/cowsay/git-r0/git
> > reset --hard HEAD'' exited with 0
> > 
> > The solution is to add new path to git alternates so that original path
> > will work outside buildchroot, and new one will work in buildchroot.
> > 
> > Signed-off-by: Uladzimir Bely <ub...@ilbers.de>
> > ---
> > meta/classes/dpkg-base.bbclass | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/meta/classes/dpkg-base.bbclass 
> b/meta/classes/dpkg-base.bbclass
> > index ec8fbc1..4b9b782 100644
> > --- a/meta/classes/dpkg-base.bbclass
> > +++ b/meta/classes/dpkg-base.bbclass
> > @@ -46,7 +46,7 @@ python do_adjust_git() {
> > 
> > if os.path.exists(alternates):
> > cmd = ["sed", "-i", alternates, "-e",
> > - "s|{}|/downloads|".format(d.getVar("DL_DIR"))]
> > + "s|\({}\(.*\)\)|\\1\\n/downloads\\2|".format(d.getVar("DL_DIR"))]
> > bb.note(' '.join(cmd))
> > if subprocess.call(cmd) != 0:
> > bb.fatal("git alternates adjustment failed")
> > 
>
> This doesn'tt work, already tried: git fails if just one record is not
> working.
>
> Jan
>
> -- 
> Siemens AG, T RDA IOT
> Corporate Competence Center Embedded Linux
>

[-- Attachment #1.2: Type: text/html, Size: 2744 bytes --]

Re: [RFC 1/2] dpkg-base: Add new git alternates record instead of replace old one

[Jan Kiszka]

On 12.08.21 11:18, Uladzimir Bely wrote:
> Yes, git still shows the message a currently invalid alternate (one of
> two records), but it's able to work further (reset --hard, etc).
> 
> But I can agree that this way could be unacceptable due this error message.
> 

Interesting - it actually failed in the past, not only complained.
Anyway, just sent my patches.

Jan

> четверг, 12 августа 2021 г. в 12:00:58 UTC+3, Jan Kiszka:
> 
>     On 12.08.21 10:10, Uladzimir Bely wrote:
>     > Task adjust_git modifies git alternates by replacing DL_DIR to
>     /downloads.
>     > New path is used inside buildchroot, so that working git repo
>     > becomes broken outside buildchroot.
>     >
>     > This causes problems with rebuilding such packages as cowsay if
>     something
>     > has changed in bitbake environment since previous build:
>     >
>     > ERROR:
>     > mc:qemuarm64-buster:cowsay-git-r0 do_patch: Command Error: 'sh -c
>     'git
>     >
>     --work-tree=/path/build/tmp/work/debian-buster-arm64/cowsay/git-r0/git
>     > reset --hard HEAD'' exited with 0
>     >
>     > The solution is to add new path to git alternates so that original
>     path
>     > will work outside buildchroot, and new one will work in buildchroot.
>     >
>     > Signed-off-by: Uladzimir Bely <ub...@ilbers.de>
>     > ---
>     > meta/classes/dpkg-base.bbclass | 2 +-
>     > 1 file changed, 1 insertion(+), 1 deletion(-)
>     >
>     > diff --git a/meta/classes/dpkg-base.bbclass
>     b/meta/classes/dpkg-base.bbclass
>     > index ec8fbc1..4b9b782 100644
>     > --- a/meta/classes/dpkg-base.bbclass
>     > +++ b/meta/classes/dpkg-base.bbclass
>     > @@ -46,7 +46,7 @@ python do_adjust_git() {
>     >
>     > if os.path.exists(alternates):
>     > cmd = ["sed", "-i", alternates, "-e",
>     > - "s|{}|/downloads|".format(d.getVar("DL_DIR"))]
>     > + "s|\({}\(.*\)\)|\\1\\n/downloads\\2|".format(d.getVar("DL_DIR"))]
>     > bb.note(' '.join(cmd))
>     > if subprocess.call(cmd) != 0:
>     > bb.fatal("git alternates adjustment failed")
>     >
> 
>     This doesn'tt work, already tried: git fails if just one record is not
>     working.
> 
>     Jan
> 
>     -- 
>     Siemens AG, T RDA IOT
>     Corporate Competence Center Embedded Linux
> 
> -- 
> You received this message because you are subscribed to the Google
> Groups "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to isar-users+unsubscribe@googlegroups.com
> <mailto:isar-users+unsubscribe@googlegroups.com>.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/isar-users/81c0fa46-a359-47bd-bed7-866671649c96n%40googlegroups.com
> <https://groups.google.com/d/msgid/isar-users/81c0fa46-a359-47bd-bed7-866671649c96n%40googlegroups.com?utm_medium=email&utm_source=footer>.

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

[RFC 2/2] dpkg-gdb: Reset git to SRCREV revision before patching

[Uladzimir Bely]

Rebuilding some packages like cowsay after ISAR configuration
changes causes error during do_patch because the patches were
already applied at previous build and git HEAD was changed.

The change resets package's git HEAD to the appropriate revision.

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 meta/classes/dpkg-gbp.bbclass | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/meta/classes/dpkg-gbp.bbclass b/meta/classes/dpkg-gbp.bbclass
index 20d2d4c..d37b07e 100644
--- a/meta/classes/dpkg-gbp.bbclass
+++ b/meta/classes/dpkg-gbp.bbclass
@@ -26,3 +26,13 @@ builddeps_install_append() {
 dpkg_runbuild_prepend() {
     export GBP_PREFIX="gbp buildpackage --git-ignore-new ${GBP_EXTRA_OPTIONS} --git-builder="
 }
+
+do_patch[prefuncs] = "set_src_rev"
+
+python set_src_rev() {
+    srcsubdir = d.getVar('S')
+    srcrev = d.getVar('SRCREV')
+
+    if srcrev:
+        bb.process.run('git reset --hard ' + srcrev, cwd=srcsubdir)
+}
-- 
2.20.1

Re: [RFC 0/2] Fixes for dpkg-gdb packages rebuild

[Jan Kiszka]

On 12.08.21 10:10, Uladzimir Bely wrote:
> Second isar build can cause some packages rebuild due to configuration
> changes and some of thems (like cowsay) fail due to broken git tree
> and/or already being applied.
> 
> Uladzimir Bely (2):
>   dpkg-base: Add new git alternates record instead of replace old one
>   dpkg-gdb: Reset git to SRCREV revision before patching
> 
>  meta/classes/dpkg-base.bbclass |  2 +-
>  meta/classes/dpkg-gbp.bbclass  | 10 ++++++++++
>  2 files changed, 11 insertions(+), 1 deletion(-)
> 

I've fixed this locally, enabling alternates both inside and outside the
buildchroot without switching it.

Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux