New mount logic breaks DEBIAN_BUILD_DEPENDS

New mount logic breaks DEBIAN_BUILD_DEPENDS

[Moessbauer, Felix]

[-- Attachment #1: Type: text/plain, Size: 902 bytes --]

Hi,

The ref 3360b7d9ff85172dc3d2275c35caec71149b35ef breaks building packages that inherit from dpkg-raw and use DEBIAN_BUILD_DEPENDS like the minimal reproducer below.
During do_install_builddeps, the mount points are not mounted (or get unmounted).
Also latest next (591a537d3cc9f3ef35e008fdff0fe631714aa68d) is affected.

inherit dpkg-raw
DESCRIPTION = "Reproduce ISAR mount issues"
DEBIAN_BUILD_DEPENDS = "htop"

Build error:
Err:4 file:/isar-apt isar/main amd64 Packages
  File not found - /isar-apt/dists/isar/main/binary-amd64/Packages (2: No such file or directory)
Get:5 file:/isar-apt isar/main Translation-en
Ign:5 file:/isar-apt isar/main Translation-en
Reading package lists...
E: Failed to fetch file:/isar-apt/dists/isar/main/binary-amd64/Packages  File not found - /isar-apt/dists/isar/main/binary-amd64/Packages (2: No such file or directory)

Best regards,
Felix

[-- Attachment #2: Type: text/html, Size: 2960 bytes --]

Re: New mount logic breaks DEBIAN_BUILD_DEPENDS

[Henning Schild]

I can confirm that after the previous problem in next was fixed i could
never get any next-based project build in CI. So far i did not have the
time to look into the issue, but Felix did.

regards,
Henning

Am Tue, 22 Jun 2021 12:45:15 +0200
schrieb "Moessbauer, Felix (T RDA IOT SES-DE)"
<felix.moessbauer@siemens.com>:

> Hi,
> 
>  
> 
> The ref 3360b7d9ff85172dc3d2275c35caec71149b35ef breaks building
> packages that inherit from dpkg-raw and use DEBIAN_BUILD_DEPENDS like
> the minimal reproducer below.
> 
> During do_install_builddeps, the mount points are not mounted (or get
> unmounted). Also latest next
> (591a537d3cc9f3ef35e008fdff0fe631714aa68d) is affected.
> 
>  
> 
> inherit dpkg-raw
> 
> DESCRIPTION = "Reproduce ISAR mount issues"
> 
> DEBIAN_BUILD_DEPENDS = "htop"
> 
>  
> 
> Build error:
> 
> Err:4 file:/isar-apt isar/main amd64 Packages
> 
>   File not found - /isar-apt/dists/isar/main/binary-amd64/Packages
> (2: No such file or directory)
> 
> Get:5 file:/isar-apt isar/main Translation-en
> 
> Ign:5 file:/isar-apt isar/main Translation-en
> 
> Reading package lists...
> 
> E: Failed to fetch
> file:/isar-apt/dists/isar/main/binary-amd64/Packages  File not found
> - /isar-apt/dists/isar/main/binary-amd64/Packages (2: No such file or
> directory)
> 
>  
> 
> Best regards,
> Felix
> 

Re: New mount logic breaks DEBIAN_BUILD_DEPENDS

[Henning Schild]

I tried it on 591a537d3cc9f3ef35e008fdff0fe631714aa68d with

bitbake mc:qemuamd64-buster:example-raw

patch

--- a/meta-isar/recipes-app/example-raw/example-raw_0.3.bb
+++ b/meta-isar/recipes-app/example-raw/example-raw_0.3.bb
@@ -6,6 +6,7 @@
 DESCRIPTION = "Sample application for ISAR"
 MAINTAINER = "Your name here <you@domain.com>"
 DEBIAN_DEPENDS = "apt (>= 0.4.2), passwd"
+DEBIAN_BUILD_DEPENDS = "htop"
 
 SRC_URI = "file://README \
           file://postinst \

bitbake mc:qemuamd64-buster:example-raw

That second bitbake worked. So not sure it is that simple.

Henning


Am Tue, 22 Jun 2021 12:45:15 +0200
schrieb "Moessbauer, Felix (T RDA IOT SES-DE)"
<felix.moessbauer@siemens.com>:

> Hi,
> 
>  
> 
> The ref 3360b7d9ff85172dc3d2275c35caec71149b35ef breaks building
> packages that inherit from dpkg-raw and use DEBIAN_BUILD_DEPENDS like
> the minimal reproducer below.
> 
> During do_install_builddeps, the mount points are not mounted (or get
> unmounted). Also latest next
> (591a537d3cc9f3ef35e008fdff0fe631714aa68d) is affected.
> 
>  
> 
> inherit dpkg-raw
> 
> DESCRIPTION = "Reproduce ISAR mount issues"
> 
> DEBIAN_BUILD_DEPENDS = "htop"
> 
>  
> 
> Build error:
> 
> Err:4 file:/isar-apt isar/main amd64 Packages
> 
>   File not found - /isar-apt/dists/isar/main/binary-amd64/Packages
> (2: No such file or directory)
> 
> Get:5 file:/isar-apt isar/main Translation-en
> 
> Ign:5 file:/isar-apt isar/main Translation-en
> 
> Reading package lists...
> 
> E: Failed to fetch
> file:/isar-apt/dists/isar/main/binary-amd64/Packages  File not found
> - /isar-apt/dists/isar/main/binary-amd64/Packages (2: No such file or
> directory)
> 
>  
> 
> Best regards,
> Felix
> 

Re: New mount logic breaks DEBIAN_BUILD_DEPENDS

[Jan Kiszka]

[please no html on mailing lists. Not only because my mail client is
broken over - according to TB devs - broken html format outlook generates]

On 22.06.21 12:45, Moessbauer, Felix (T RDA IOT SES-DE) wrote:
> Hi,
> 
>  
> 
> The ref 3360b7d9ff85172dc3d2275c35caec71149b35ef breaks building
> packages that inherit from dpkg-raw and use DEBIAN_BUILD_DEPENDS like
> the minimal reproducer below.
> 
> During do_install_builddeps, the mount points are not mounted (or get
> unmounted).
> Also latest next (591a537d3cc9f3ef35e008fdff0fe631714aa68d) is affected.
> 
>  
> 
> inherit dpkg-raw
> 
> DESCRIPTION = "Reproduce ISAR mount issues"
> 
> DEBIAN_BUILD_DEPENDS = "htop"
> 
>  
> 
> Build error:
> 
> Err:4 file:/isar-apt isar/main amd64 Packages
> 
>   File not found - /isar-apt/dists/isar/main/binary-amd64/Packages (2:
> No such file or directory)
> 
> Get:5 file:/isar-apt isar/main Translation-en
> 
> Ign:5 file:/isar-apt isar/main Translation-en
> 
> Reading package lists...
> 
> E: Failed to fetch file:/isar-apt/dists/isar/main/binary-amd64/Packages 
> File not found - /isar-apt/dists/isar/main/binary-amd64/Packages (2: No
> such file or directory)
> 

I've see this error with a complex build yesterday as well. Was only
resolves by complete purging of build results.

I would strongly suggest to revert this series and rework it carefully,
even more as it was not fixing any kind of urgent issues (we were living
with the previous behavior for many years now).

Jan

-- 
Siemens AG, T RDA IOT
Corporate Competence Center Embedded Linux

Re: New mount logic breaks DEBIAN_BUILD_DEPENDS

[Baurzhan Ismagulov]

On Wed, Jun 23, 2021 at 07:41:47AM +0200, Jan Kiszka wrote:
> I've see this error with a complex build yesterday as well. Was only
> resolves by complete purging of build results.
> 
> I would strongly suggest to revert this series and rework it carefully,
> even more as it was not fixing any kind of urgent issues (we were living
> with the previous behavior for many years now).

The series was created due to severe "Couldn't unmount, retrying..." problems
in our CI. We'll check whether it occurs with the series reverted and let you
know.

The old behavior worked because the directories were either unmounted lazily,
or unmounted at the very end. Unaddressed, that would block cleanroom rebuilds.
I hope we can find a practical way to handle this. We'll continue trying to
reproduce your issue and also check this one.

With kind regards,
Baurzhan.

Re: New mount logic breaks DEBIAN_BUILD_DEPENDS

[Anton Mikanovich]

[-- Attachment #1: Type: text/plain, Size: 2120 bytes --]

I have an idea about possible reason of this issue. Did you experience 
build fail before that?
Reference counters are not cleared in case unsuccessful build happens.

Please look at `[PATCH] mount: Cleanup reference counters before build` 
just pushed to the maillist.

22.06.2021 13:45, Moessbauer, Felix wrote:
>
> Hi,
>
> The ref 3360b7d9ff85172dc3d2275c35caec71149b35ef breaks building 
> packages that inherit from dpkg-raw and use DEBIAN_BUILD_DEPENDS like 
> the minimal reproducer below.
>
> During do_install_builddeps, the mount points are not mounted (or get 
> unmounted).
> Also latest next (591a537d3cc9f3ef35e008fdff0fe631714aa68d) is affected.
>
> inherit dpkg-raw
>
> DESCRIPTION = "Reproduce ISAR mount issues"
>
> DEBIAN_BUILD_DEPENDS = "htop"
>
> Build error:
>
> Err:4 file:/isar-apt isar/main amd64 Packages
>
>   File not found - /isar-apt/dists/isar/main/binary-amd64/Packages (2: 
> No such file or directory)
>
> Get:5 file:/isar-apt isar/main Translation-en
>
> Ign:5 file:/isar-apt isar/main Translation-en
>
> Reading package lists...
>
> E: Failed to fetch 
> file:/isar-apt/dists/isar/main/binary-amd64/Packages  File not found - 
> /isar-apt/dists/isar/main/binary-amd64/Packages (2: No such file or 
> directory)
>
> Best regards,
> Felix
>
> -- 
> You received this message because you are subscribed to the Google 
> Groups "isar-users" group.
> To unsubscribe from this group and stop receiving emails from it, send 
> an email to isar-users+unsubscribe@googlegroups.com 
> <mailto:isar-users+unsubscribe@googlegroups.com>.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/isar-users/AM9PR10MB4869CA345FC584A4ABB5219B89099%40AM9PR10MB4869.EURPRD10.PROD.OUTLOOK.COM 
> <https://groups.google.com/d/msgid/isar-users/AM9PR10MB4869CA345FC584A4ABB5219B89099%40AM9PR10MB4869.EURPRD10.PROD.OUTLOOK.COM?utm_medium=email&utm_source=footer>.

-- 
Anton Mikanovich
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov


[-- Attachment #2: Type: text/html, Size: 4676 bytes --]

RE: New mount logic breaks DEBIAN_BUILD_DEPENDS

[Moessbauer, Felix]

Hi,

These issues also appear in "from-scratch" builds.
In our case, we discovered it in the CI of a project that uses ISAR after bumping the version.
At least in our use-case, it always fails (it is deterministic).

I'm going to try the mentioned patch and report back.

Best regards,
Felix

From: Anton Mikanovich <amikan@ilbers.de> 
Sent: Wednesday, June 23, 2021 2:00 PM
To: Moessbauer, Felix (T RDA IOT SES-DE) <felix.moessbauer@siemens.com>; isar-users <isar-users@googlegroups.com>
Cc: Schild, Henning (T RDA IOT SES-DE) <henning.schild@siemens.com>; Kiszka, Jan (T RDA IOT) <jan.kiszka@siemens.com>
Subject: Re: New mount logic breaks DEBIAN_BUILD_DEPENDS

I have an idea about possible reason of this issue. Did you experience build fail before that?
Reference counters are not cleared in case unsuccessful build happens.

Please look at `[PATCH] mount: Cleanup reference counters before build` just pushed to the maillist.
22.06.2021 13:45, Moessbauer, Felix wrote:
Hi,
 
The ref 3360b7d9ff85172dc3d2275c35caec71149b35ef breaks building packages that inherit from dpkg-raw and use DEBIAN_BUILD_DEPENDS like the minimal reproducer below.
During do_install_builddeps, the mount points are not mounted (or get unmounted).
Also latest next (591a537d3cc9f3ef35e008fdff0fe631714aa68d) is affected.
 
inherit dpkg-raw
DESCRIPTION = "Reproduce ISAR mount issues"
DEBIAN_BUILD_DEPENDS = "htop"
 
Build error:
Err:4 file:/isar-apt isar/main amd64 Packages
  File not found - /isar-apt/dists/isar/main/binary-amd64/Packages (2: No such file or directory)
Get:5 file:/isar-apt isar/main Translation-en
Ign:5 file:/isar-apt isar/main Translation-en
Reading package lists...
E: Failed to fetch file:/isar-apt/dists/isar/main/binary-amd64/Packages  File not found - /isar-apt/dists/isar/main/binary-amd64/Packages (2: No such file or directory)
 
Best regards,
Felix
-- 
You received this message because you are subscribed to the Google Groups "isar-users" group.
To unsubscribe from this group and stop receiving emails from it, send an email to mailto:isar-users+unsubscribe@googlegroups.com.
To view this discussion on the web visit https://eur01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgroups.google.com%2Fd%2Fmsgid%2Fisar-users%2FAM9PR10MB4869CA345FC584A4ABB5219B89099%2540AM9PR10MB4869.EURPRD10.PROD.OUTLOOK.COM%3Futm_medium%3Demail%26utm_source%3Dfooter&data=04%7C01%7Cfelix.moessbauer%40siemens.com%7Cab22a984d2e340f11c3d08d9363e798e%7C38ae3bcd95794fd4addab42e1495d55a%7C1%7C0%7C637600464237426405%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=v1%2FQzG4eQ5WtSRKZTYaoJRLdt%2FaWCD9lDOcm8LAA1iM%3D&reserved=0.
-- 
Anton Mikanovich
Promwad Ltd.
External service provider of ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn, Germany
+49 (89) 122 67 24-0
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov