Re: [PATCH] Consolidate over PARALLEL_MAKE

[Jan Kiszka <jan.kiszka@siemens.com>]

On 03.06.20 11:49, Henning Schild wrote:
> On Wed, 3 Jun 2020 08:37:57 +0200
> "[ext] Jan Kiszka" <jan.kiszka@siemens.com> wrote:
> 
>> From: Jan Kiszka <jan.kiszka@siemens.com>
>>
>> Replace self-invented / open-coded parallel build rules over the OE
>> pattern PARALLEL_MAKE. Should specifically reduce the spread of
>> PARALLEL_BUILD definitions in downstream layers. Mea culpa.
>>
>> Signed-off-by: Jan Kiszka <jan.kiszka@siemens.com>
>> ---
>>  meta-isar/recipes-bsp/opensbi/files/sifive-fu540-rules | 5 ++---
>>  meta/classes/dpkg.bbclass                              | 1 +
>>  meta/conf/bitbake.conf                                 | 4 ++++
>>  meta/recipes-bsp/u-boot/files/debian/rules             | 8 +++-----
>>  meta/recipes-kernel/linux/files/debian/isar/build.tmpl | 2 +-
>>  5 files changed, 11 insertions(+), 9 deletions(-)
>>
>> diff --git a/meta-isar/recipes-bsp/opensbi/files/sifive-fu540-rules
>> b/meta-isar/recipes-bsp/opensbi/files/sifive-fu540-rules index
>> c8feb715..c3fc50a9 100644 ---
>> a/meta-isar/recipes-bsp/opensbi/files/sifive-fu540-rules +++
>> b/meta-isar/recipes-bsp/opensbi/files/sifive-fu540-rules @@ -4,10
>> +4,9 @@ ifneq ($(DEB_BUILD_GNU_TYPE),$(DEB_HOST_GNU_TYPE)) export
>> CROSS_COMPILE=$(DEB_HOST_GNU_TYPE)- endif
>>  
>> -PARALLEL_BUILD=-j $(shell echo $$(($$(nproc) * 2)))
>> -
>>  override_dh_auto_build:
>> -	CFLAGS= LDFLAGS= $(MAKE) $(PARALLEL_BUILD)
>> PLATFORM=sifive/fu540
>> FW_PAYLOAD_PATH=/usr/lib/u-boot/sifive_fu540/u-boot.bin
>> +	CFLAGS= LDFLAGS= $(MAKE) $(PARALLEL_MAKE)
>> PLATFORM=sifive/fu540 \
>> +
>> FW_PAYLOAD_PATH=/usr/lib/u-boot/sifive_fu540/u-boot.bin 
>>  %:
>>  	dh $@
>> diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass
>> index 585365ee..acccc1f2 100644
>> --- a/meta/classes/dpkg.bbclass
>> +++ b/meta/classes/dpkg.bbclass
>> @@ -25,6 +25,7 @@ addtask devshell after do_install_builddeps
>>  # Build package from sources using build script
>>  dpkg_runbuild() {
>>      E="${@ isar_export_proxies(d)}"
>> +    export PARALLEL_MAKE="${PARALLEL_MAKE}"
>>      sudo -E chroot --userspec=$( id -u ):$( id -g )
>> ${BUILDCHROOT_DIR} \ /isar/build.sh ${PP}/${PPS} ${DISTRO_ARCH}
>>  }
>> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf
>> index 848e085a..0e0c79e0 100644
>> --- a/meta/conf/bitbake.conf
>> +++ b/meta/conf/bitbake.conf
>> @@ -101,8 +101,12 @@ BB_HASHCONFIG_WHITELIST ?=
>> "${BB_HASHBASE_WHITELIST} DATE TIME SSH_AGENT_PID \
>> BB_SIGNATURE_HANDLER ?= "basichash" BB_STAMP_POLICY ?= "full"
>>  
>> +# Default to setting automatically based on cpu count
>>  BB_NUMBER_THREADS ?= "${@bb.utils.cpu_count()}"
>>  
>> +# Default to setting automatically based on cpu count
>> +PARALLEL_MAKE ?= "-j ${@bb.utils.cpu_count()}"
> 
> Maybe derive it from the other one "-j ${BB_NUMBER_THREADS}" ?
> 

I'm just using the OE pattern here.

> For make it is often suggested to use slightly more threads than
> available CPUs, when some threads are doing IO you still get to use all
> CPUs. There was a "*2" ...

I know, but same reasoning as above.

And given that cpu_count is cores*threads with, typically, threads >= 2,
this overcommittment is less important today.

Jan

-- 
Siemens AG, Corporate Technology, CT RDA IOT SES-DE
Corporate Competence Center Embedded Linux