[PATCH] linux-custom: Add lock to dependency installation

[PATCH] linux-custom: Add lock to dependency installation

[Jan Kiszka]

From: Jan Kiszka <jan.kiszka@siemens.com>

The build script of linux-custom requires the same lock protection as
build.sh of the dpkg builder class. Otherwise, we see the infamous build
failures again due to contention on the dpkg database lock.

Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
---
 meta/recipes-kernel/linux/files/build-kernel.sh | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh
index 94897ec..42c8ff6 100644
--- a/meta/recipes-kernel/linux/files/build-kernel.sh
+++ b/meta/recipes-kernel/linux/files/build-kernel.sh
@@ -13,8 +13,10 @@ REPACK_DIR="$1/../repack"
 REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image"
 REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers"
 
-apt-get install -y -o Debug::pkgProblemResolver=yes --no-install-recommends \
-	${KBUILD_DEPENDS}
+# Lock-protected because apt and dpkg do not wait in case of contention
+flock /dpkg.lock \
+    apt-get install -y -o Debug::pkgProblemResolver=yes \
+        --no-install-recommends ${KBUILD_DEPENDS}
 
 cd $1
 make olddefconfig
-- 
2.13.6

Re: [PATCH] linux-custom: Add lock to dependency installation

[Jan Kiszka]

On 2018-02-22 22:25, [ext] Jan Kiszka wrote:
> From: Jan Kiszka <jan.kiszka@siemens.com>
> 
> The build script of linux-custom requires the same lock protection as
> build.sh of the dpkg builder class. Otherwise, we see the infamous build
> failures again due to contention on the dpkg database lock.
> 
> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
>  meta/recipes-kernel/linux/files/build-kernel.sh | 6 ++++--
>  1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh
> index 94897ec..42c8ff6 100644
> --- a/meta/recipes-kernel/linux/files/build-kernel.sh
> +++ b/meta/recipes-kernel/linux/files/build-kernel.sh
> @@ -13,8 +13,10 @@ REPACK_DIR="$1/../repack"
>  REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image"
>  REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers"
>  
> -apt-get install -y -o Debug::pkgProblemResolver=yes --no-install-recommends \
> -	${KBUILD_DEPENDS}
> +# Lock-protected because apt and dpkg do not wait in case of contention
> +flock /dpkg.lock \
> +    apt-get install -y -o Debug::pkgProblemResolver=yes \
> +        --no-install-recommends ${KBUILD_DEPENDS}
>  
>  cd $1
>  make olddefconfig
> 

Could you have a look at this fix? It's essential when building custom
kernels and it's needed to reduce my queue to 0.

Thanks
Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH] linux-custom: Add lock to dependency installation

[Alexander Smirnov]

On 02/23/2018 12:25 AM, Jan Kiszka wrote:
> From: Jan Kiszka <jan.kiszka@siemens.com>
> 
> The build script of linux-custom requires the same lock protection as
> build.sh of the dpkg builder class. Otherwise, we see the infamous build
> failures again due to contention on the dpkg database lock.
> 
> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
> ---
>   meta/recipes-kernel/linux/files/build-kernel.sh | 6 ++++--
>   1 file changed, 4 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh b/meta/recipes-kernel/linux/files/build-kernel.sh
> index 94897ec..42c8ff6 100644
> --- a/meta/recipes-kernel/linux/files/build-kernel.sh
> +++ b/meta/recipes-kernel/linux/files/build-kernel.sh
> @@ -13,8 +13,10 @@ REPACK_DIR="$1/../repack"
>   REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image"
>   REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers"
>   
> -apt-get install -y -o Debug::pkgProblemResolver=yes --no-install-recommends \
> -	${KBUILD_DEPENDS}
> +# Lock-protected because apt and dpkg do not wait in case of contention
> +flock /dpkg.lock \
> +    apt-get install -y -o Debug::pkgProblemResolver=yes \
> +        --no-install-recommends ${KBUILD_DEPENDS}

The same stuff as in "meta/recipes-devtools/buildchroot/files/build.sh", 
but different style. Can we keep things similar as much as possible?

Why not

(
    ...
) 42>/dpkg.lock

?

Alex

Re: [PATCH] linux-custom: Add lock to dependency installation

[Jan Kiszka]

On 2018-03-05 08:16, Alexander Smirnov wrote:
> 
> 
> On 02/23/2018 12:25 AM, Jan Kiszka wrote:
>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>
>> The build script of linux-custom requires the same lock protection as
>> build.sh of the dpkg builder class. Otherwise, we see the infamous build
>> failures again due to contention on the dpkg database lock.
>>
>> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
>> ---
>>   meta/recipes-kernel/linux/files/build-kernel.sh | 6 ++++--
>>   1 file changed, 4 insertions(+), 2 deletions(-)
>>
>> diff --git a/meta/recipes-kernel/linux/files/build-kernel.sh
>> b/meta/recipes-kernel/linux/files/build-kernel.sh
>> index 94897ec..42c8ff6 100644
>> --- a/meta/recipes-kernel/linux/files/build-kernel.sh
>> +++ b/meta/recipes-kernel/linux/files/build-kernel.sh
>> @@ -13,8 +13,10 @@ REPACK_DIR="$1/../repack"
>>   REPACK_LINUX_IMAGE_DIR="${REPACK_DIR}/linux-image"
>>   REPACK_LINUX_HEADERS_DIR="${REPACK_DIR}/linux-headers"
>>   -apt-get install -y -o Debug::pkgProblemResolver=yes
>> --no-install-recommends \
>> -    ${KBUILD_DEPENDS}
>> +# Lock-protected because apt and dpkg do not wait in case of contention
>> +flock /dpkg.lock \
>> +    apt-get install -y -o Debug::pkgProblemResolver=yes \
>> +        --no-install-recommends ${KBUILD_DEPENDS}
> 
> The same stuff as in "meta/recipes-devtools/buildchroot/files/build.sh",
> but different style. Can we keep things similar as much as possible?
> 
> Why not
> 
> (
>    ...
> ) 42>/dpkg.lock
> 
> ?

This is

if (x)
	y;

vs.

if (x) {
	y;
	z;
}

See also the man page of flock.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux

Re: [PATCH] linux-custom: Add lock to dependency installation

[Alexander Smirnov]

On 02/23/2018 12:25 AM, Jan Kiszka wrote:
> From: Jan Kiszka <jan.kiszka@siemens.com>
> 
> The build script of linux-custom requires the same lock protection as
> build.sh of the dpkg builder class. Otherwise, we see the infamous build
> failures again due to contention on the dpkg database lock.
> 

Applied to next, thanks.

Alex