[PATCH] buildchroot: Perform upgrade after build dependency installation

[PATCH] buildchroot: Perform upgrade after build dependency installation

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

When we partially rebuild after updating the version of a build
dependency of another package, we do not properly upgrade that
dependency in buildchroot. The reason is that "apt-get install"
performs upgrades only for the explicitly listed packages. But we
install build dependencies indirectly, we a meta package's dependency.

Resolve that be running an explicit "apt-get upgrade" after the
build dependency installation. This will ensure pulling the latest
versions for isar-apt.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 meta/recipes-devtools/buildchroot/files/deps.sh | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/meta/recipes-devtools/buildchroot/files/deps.sh b/meta/recipes-devtools/buildchroot/files/deps.sh
index 4bd604f..2501f05 100644
--- a/meta/recipes-devtools/buildchroot/files/deps.sh
+++ b/meta/recipes-devtools/buildchroot/files/deps.sh
@@ -25,3 +25,6 @@ apt-get update \

 # Install all build deps
 mk-build-deps $set_arch -t "${install_cmd}" -i -r debian/control
+
+# Upgrade any already installed packages in case we are partially rebuilding
+apt-get upgrade -y
--
2.16.4

Re: [PATCH] buildchroot: Perform upgrade after build dependency installation

[Henning Schild]

Am Tue, 22 Jan 2019 17:22:14 +0100
schrieb Jan Kiszka <jan.kiszka@web.de>:

> From: Jan Kiszka <jan.kiszka@siemens.com>
> 
> When we partially rebuild after updating the version of a build
> dependency of another package, we do not properly upgrade that
> dependency in buildchroot. The reason is that "apt-get install"
> performs upgrades only for the explicitly listed packages. But we
> install build dependencies indirectly, we a meta package's dependency.
> 
> Resolve that be running an explicit "apt-get upgrade" after the
> build dependency installation. This will ensure pulling the latest
> versions for isar-apt.

I am not sure i fully understand the problem given your description,
but i get the idea.

The mk-build-deps calls an apt-get that will not do anything even if
builddep-0.1 has to be updated to builddep-0.2? Maybe you can address
the problem with the "-t" argument of mk-build-dep?

Henning

> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
>  meta/recipes-devtools/buildchroot/files/deps.sh | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/meta/recipes-devtools/buildchroot/files/deps.sh
> b/meta/recipes-devtools/buildchroot/files/deps.sh index
> 4bd604f..2501f05 100644 ---
> a/meta/recipes-devtools/buildchroot/files/deps.sh +++
> b/meta/recipes-devtools/buildchroot/files/deps.sh @@ -25,3 +25,6 @@
> apt-get update \
> 
>  # Install all build deps
>  mk-build-deps $set_arch -t "${install_cmd}" -i -r debian/control
> +
> +# Upgrade any already installed packages in case we are partially
> rebuilding +apt-get upgrade -y
> --
> 2.16.4
> 

Re: [PATCH] buildchroot: Perform upgrade after build dependency installation

[Jan Kiszka]

On 22.01.19 18:08, Henning Schild wrote:
> Am Tue, 22 Jan 2019 17:22:14 +0100
> schrieb Jan Kiszka <jan.kiszka@web.de>:
>
>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>
>> When we partially rebuild after updating the version of a build
>> dependency of another package, we do not properly upgrade that
>> dependency in buildchroot. The reason is that "apt-get install"
>> performs upgrades only for the explicitly listed packages. But we
>> install build dependencies indirectly, we a meta package's dependency.
>>
>> Resolve that be running an explicit "apt-get upgrade" after the
>> build dependency installation. This will ensure pulling the latest
>> versions for isar-apt.
>
> I am not sure i fully understand the problem given your description,
> but i get the idea.
>
> The mk-build-deps calls an apt-get that will not do anything even if
> builddep-0.1 has to be updated to builddep-0.2? Maybe you can address

If you refer with "builddep" to the meta package that mk-build-dep is generating
- no, it's one of the indirect dependencies it refers to. Concrete case: kernel
update, and kernel-headers will not be upgraded, thus the module built against
the wrong kernel.

> the problem with the "-t" argument of mk-build-dep?

I've tried various switched to apt-get install, but it just reported to not do
the identified upgrades. I would prefer a magic switch as well, so I'm all ears
which one we may miss.

Jan

Re: [PATCH] buildchroot: Perform upgrade after build dependency installation

[Henning Schild]

Am Tue, 22 Jan 2019 19:19:58 +0100
schrieb Jan Kiszka <jan.kiszka@web.de>:

> On 22.01.19 18:08, Henning Schild wrote:
> > Am Tue, 22 Jan 2019 17:22:14 +0100
> > schrieb Jan Kiszka <jan.kiszka@web.de>:
> >  
> >> From: Jan Kiszka <jan.kiszka@siemens.com>
> >>
> >> When we partially rebuild after updating the version of a build
> >> dependency of another package, we do not properly upgrade that
> >> dependency in buildchroot. The reason is that "apt-get install"
> >> performs upgrades only for the explicitly listed packages. But we
> >> install build dependencies indirectly, we a meta package's
> >> dependency.
> >>
> >> Resolve that be running an explicit "apt-get upgrade" after the
> >> build dependency installation. This will ensure pulling the latest
> >> versions for isar-apt.  
> >
> > I am not sure i fully understand the problem given your description,
> > but i get the idea.
> >
> > The mk-build-deps calls an apt-get that will not do anything even if
> > builddep-0.1 has to be updated to builddep-0.2? Maybe you can
> > address  
> 
> If you refer with "builddep" to the meta package that mk-build-dep is
> generating
> - no, it's one of the indirect dependencies it refers to. Concrete
> case: kernel update, and kernel-headers will not be upgraded, thus
> the module built against the wrong kernel.

No i am talking about the package at the lower end. We have A -> B ->
C0 + C1 + C2. A is the one we want to build, B is the artificial
package, and Cxx are the ones on the "lower end". 

> > the problem with the "-t" argument of mk-build-dep?  
> 
> I've tried various switched to apt-get install, but it just reported
> to not do the identified upgrades. I would prefer a magic switch as
> well, so I'm all ears which one we may miss.

Did the debian version string change to something greater? If not it
should probably increase.
Another option would be to "apt-get remove" a packages from all
buildchroots on its clean, unless it was explicitly installed.

But i guess what you came up with is fine as well, and less
complicated.

Henning 

> Jan

Re: [PATCH] buildchroot: Perform upgrade after build dependency installation

[Maxim Yu. Osipov]

On 1/22/19 5:22 PM, Jan Kiszka wrote:
> From: Jan Kiszka <jan.kiszka@siemens.com>
> 
> When we partially rebuild after updating the version of a build
> dependency of another package, we do not properly upgrade that
> dependency in buildchroot. The reason is that "apt-get install"
> performs upgrades only for the explicitly listed packages. But we
> install build dependencies indirectly, we a meta package's dependency.
> 
> Resolve that be running an explicit "apt-get upgrade" after the
> build dependency installation. This will ensure pulling the latest
> versions for isar-apt.

When ran ci_build.sh -q -f build failed on known problem:

| The lock file 
'/workspace/build/isar_mosipov_develop/17/build/downloads/base-apt/db/debian/lockfile' 
already exists. There might be another instance with the
| same database dir running. To avoid locking overhead, only one process
| can access the database at the same time. Do not delete the lock file 
unless
| you are sure no other version is still running!
| There have been errors!

See log:
http://isar-build.org:8080/job/isar_mosipov_develop/17/console

Investigating the problem.

Maxim.

> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
>   meta/recipes-devtools/buildchroot/files/deps.sh | 3 +++
>   1 file changed, 3 insertions(+)
> 
> diff --git a/meta/recipes-devtools/buildchroot/files/deps.sh b/meta/recipes-devtools/buildchroot/files/deps.sh
> index 4bd604f..2501f05 100644
> --- a/meta/recipes-devtools/buildchroot/files/deps.sh
> +++ b/meta/recipes-devtools/buildchroot/files/deps.sh
> @@ -25,3 +25,6 @@ apt-get update \
> 
>   # Install all build deps
>   mk-build-deps $set_arch -t "${install_cmd}" -i -r debian/control
> +
> +# Upgrade any already installed packages in case we are partially rebuilding
> +apt-get upgrade -y
> --
> 2.16.4
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov

Re: [PATCH] buildchroot: Perform upgrade after build dependency installation

[Jan Kiszka]

On 30.01.19 12:49, Maxim Yu. Osipov wrote:
> On 1/22/19 5:22 PM, Jan Kiszka wrote:
>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>
>> When we partially rebuild after updating the version of a build
>> dependency of another package, we do not properly upgrade that
>> dependency in buildchroot. The reason is that "apt-get install"
>> performs upgrades only for the explicitly listed packages. But we
>> install build dependencies indirectly, we a meta package's dependency.
>>
>> Resolve that be running an explicit "apt-get upgrade" after the
>> build dependency installation. This will ensure pulling the latest
>> versions for isar-apt.
> 
> When ran ci_build.sh -q -f build failed on known problem:
> 
> | The lock file 
> '/workspace/build/isar_mosipov_develop/17/build/downloads/base-apt/db/debian/lockfile' 
> already exists. There might be another instance with the
> | same database dir running. To avoid locking overhead, only one process
> | can access the database at the same time. Do not delete the lock file unless
> | you are sure no other version is still running!
> | There have been errors!
> 
> See log:
> http://isar-build.org:8080/job/isar_mosipov_develop/17/console
> 
> Investigating the problem.

That is what I reported to the list already [1]. Unrelated to this commit, but 
if it helps to reproduce the issue reliably, that would be helpful.

Jan

[1] https://groups.google.com/forum/#!msg/isar-users/ElRP1h_9m9I/rt6yibJuEQAJ

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH] buildchroot: Perform upgrade after build dependency installation

[Maxim Yu. Osipov]

On 1/22/19 5:22 PM, Jan Kiszka wrote:
> From: Jan Kiszka <jan.kiszka@siemens.com>
> 
> When we partially rebuild after updating the version of a build
> dependency of another package, we do not properly upgrade that
> dependency in buildchroot. The reason is that "apt-get install"
> performs upgrades only for the explicitly listed packages. But we
> install build dependencies indirectly, we a meta package's dependency.
> 
> Resolve that be running an explicit "apt-get upgrade" after the
> build dependency installation. This will ensure pulling the latest
> versions for isar-apt.

Applied to the 'next'.

Thanks,
Maxim.

> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
>   meta/recipes-devtools/buildchroot/files/deps.sh | 3 +++
>   1 file changed, 3 insertions(+)
> 
> diff --git a/meta/recipes-devtools/buildchroot/files/deps.sh b/meta/recipes-devtools/buildchroot/files/deps.sh
> index 4bd604f..2501f05 100644
> --- a/meta/recipes-devtools/buildchroot/files/deps.sh
> +++ b/meta/recipes-devtools/buildchroot/files/deps.sh
> @@ -25,3 +25,6 @@ apt-get update \
> 
>   # Install all build deps
>   mk-build-deps $set_arch -t "${install_cmd}" -i -r debian/control
> +
> +# Upgrade any already installed packages in case we are partially rebuilding
> +apt-get upgrade -y
> --
> 2.16.4
> 


-- 
Maxim Osipov
ilbers GmbH
Maria-Merian-Str. 8
85521 Ottobrunn
Germany
+49 (151) 6517 6917
mosipov@ilbers.de
http://ilbers.de/
Commercial register Munich, HRB 214197
General Manager: Baurzhan Ismagulov