From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6834007135866585088 X-Received: by 2002:adf:80eb:: with SMTP id 98mr13560386wrl.351.1591178215058; Wed, 03 Jun 2020 02:56:55 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com Received: by 2002:adf:e510:: with SMTP id j16ls2122711wrm.2.gmail; Wed, 03 Jun 2020 02:56:54 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyJaJr4RmfDFcBX26aJZ+l8e14YqoxuPuP764cVcL21U8aHNRjDZlW0zhe//PvfKbXzAbbp X-Received: by 2002:adf:9c12:: with SMTP id f18mr14548153wrc.105.1591178214463; Wed, 03 Jun 2020 02:56:54 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1591178214; cv=none; d=google.com; s=arc-20160816; b=VJIFcCj4WjkZVlNupOQOV8MYW/0s26eZfa/+2hkNoeDwb7zBmLUBBug9Hn0VvBSo58 ssPSzYHYHSDpCJnJlfpyisBQflhYAyx68PYWZ6mdKOdGP8P0XnQG5TyyWcmgvgA/RmY8 LVoYjvZRpBpIL5OWTwDP2y7HI2uKWXkddmKRRpkaASu4ka0hAANEskY7lxfNX1Jci1KV 8dSUrPRDLpzK+SmPIeUfZNcyPsi5Fg+EUDPUOVMXAeQ97vneEpX8m1Qwj8rekpX8SphS RSCRna9UFzEjAK1FGAunHzkNSpxhnsaHPApcYxHXUt1ZrQ1SRX5Z2tnGRckHXfUuby4e aNIA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject; bh=ePzxu+9zYoop/ZSF9tOMflvd2y7sFn1cXmq+rzT52/E=; b=Mg4gSF3GCtbrJLBq4t+kci5+uli9RMUlpZYHDPZYLJTgqSjDwj5Ajd0rscCj1HkvsO ZX/CEiuX5KO7fb9YMEZkmXsFB/MSL0e01rG2Zn+D2Xh9SPCAxQW3TIQC3oyDjshqZkjF dIMygdE1N5Jz09dFBYrV9WuPdTcAynKanUhyQfopT3OY/KpvZ629DtFsjUNZhCn2Nry1 z2S90ngZtKLDX1hqHIGfZFy9KkEMsBAZJ/nqTbvZTtD+D3kqrKvv/BxJ4/t3PGuHc8Om W4jSSCCXKZKiKCuDejb/4HWVpqbZd4Qd20ycifbFjf5jKm6hnY9W04eZnST5EPeWgvJM Tr0A== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Return-Path: Received: from goliath.siemens.de (goliath.siemens.de. [192.35.17.28]) by gmr-mx.google.com with ESMTPS id g7si64383wma.1.2020.06.03.02.56.54 for (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Wed, 03 Jun 2020 02:56:54 -0700 (PDT) Received-SPF: pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) client-ip=192.35.17.28; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of jan.kiszka@siemens.com designates 192.35.17.28 as permitted sender) smtp.mailfrom=jan.kiszka@siemens.com; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=siemens.com Received: from mail2.sbs.de (mail2.sbs.de [192.129.41.66]) by goliath.siemens.de (8.15.2/8.15.2) with ESMTPS id 0539urrW019816 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK) for ; Wed, 3 Jun 2020 11:56:54 +0200 Received: from [139.22.36.189] ([139.22.36.189]) by mail2.sbs.de (8.15.2/8.15.2) with ESMTP id 0539ur5p024779; Wed, 3 Jun 2020 11:56:53 +0200 Subject: Re: [PATCH] Consolidate over PARALLEL_MAKE To: Henning Schild Cc: isar-users References: <20200603114946.3f1db38c@md1za8fc.ad001.siemens.net> From: Jan Kiszka Message-ID: <6ac7fc21-55d6-8e56-695d-32a7101f970a@siemens.com> Date: Wed, 3 Jun 2020 11:56:53 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.8.0 MIME-Version: 1.0 In-Reply-To: <20200603114946.3f1db38c@md1za8fc.ad001.siemens.net> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: eDwJ2/kZOGPO On 03.06.20 11:49, Henning Schild wrote: > On Wed, 3 Jun 2020 08:37:57 +0200 > "[ext] Jan Kiszka" wrote: > >> From: Jan Kiszka >> >> Replace self-invented / open-coded parallel build rules over the OE >> pattern PARALLEL_MAKE. Should specifically reduce the spread of >> PARALLEL_BUILD definitions in downstream layers. Mea culpa. >> >> Signed-off-by: Jan Kiszka >> --- >> meta-isar/recipes-bsp/opensbi/files/sifive-fu540-rules | 5 ++--- >> meta/classes/dpkg.bbclass | 1 + >> meta/conf/bitbake.conf | 4 ++++ >> meta/recipes-bsp/u-boot/files/debian/rules | 8 +++----- >> meta/recipes-kernel/linux/files/debian/isar/build.tmpl | 2 +- >> 5 files changed, 11 insertions(+), 9 deletions(-) >> >> diff --git a/meta-isar/recipes-bsp/opensbi/files/sifive-fu540-rules >> b/meta-isar/recipes-bsp/opensbi/files/sifive-fu540-rules index >> c8feb715..c3fc50a9 100644 --- >> a/meta-isar/recipes-bsp/opensbi/files/sifive-fu540-rules +++ >> b/meta-isar/recipes-bsp/opensbi/files/sifive-fu540-rules @@ -4,10 >> +4,9 @@ ifneq ($(DEB_BUILD_GNU_TYPE),$(DEB_HOST_GNU_TYPE)) export >> CROSS_COMPILE=$(DEB_HOST_GNU_TYPE)- endif >> >> -PARALLEL_BUILD=-j $(shell echo $$(($$(nproc) * 2))) >> - >> override_dh_auto_build: >> - CFLAGS= LDFLAGS= $(MAKE) $(PARALLEL_BUILD) >> PLATFORM=sifive/fu540 >> FW_PAYLOAD_PATH=/usr/lib/u-boot/sifive_fu540/u-boot.bin >> + CFLAGS= LDFLAGS= $(MAKE) $(PARALLEL_MAKE) >> PLATFORM=sifive/fu540 \ >> + >> FW_PAYLOAD_PATH=/usr/lib/u-boot/sifive_fu540/u-boot.bin >> %: >> dh $@ >> diff --git a/meta/classes/dpkg.bbclass b/meta/classes/dpkg.bbclass >> index 585365ee..acccc1f2 100644 >> --- a/meta/classes/dpkg.bbclass >> +++ b/meta/classes/dpkg.bbclass >> @@ -25,6 +25,7 @@ addtask devshell after do_install_builddeps >> # Build package from sources using build script >> dpkg_runbuild() { >> E="${@ isar_export_proxies(d)}" >> + export PARALLEL_MAKE="${PARALLEL_MAKE}" >> sudo -E chroot --userspec=$( id -u ):$( id -g ) >> ${BUILDCHROOT_DIR} \ /isar/build.sh ${PP}/${PPS} ${DISTRO_ARCH} >> } >> diff --git a/meta/conf/bitbake.conf b/meta/conf/bitbake.conf >> index 848e085a..0e0c79e0 100644 >> --- a/meta/conf/bitbake.conf >> +++ b/meta/conf/bitbake.conf >> @@ -101,8 +101,12 @@ BB_HASHCONFIG_WHITELIST ?= >> "${BB_HASHBASE_WHITELIST} DATE TIME SSH_AGENT_PID \ >> BB_SIGNATURE_HANDLER ?= "basichash" BB_STAMP_POLICY ?= "full" >> >> +# Default to setting automatically based on cpu count >> BB_NUMBER_THREADS ?= "${@bb.utils.cpu_count()}" >> >> +# Default to setting automatically based on cpu count >> +PARALLEL_MAKE ?= "-j ${@bb.utils.cpu_count()}" > > Maybe derive it from the other one "-j ${BB_NUMBER_THREADS}" ? > I'm just using the OE pattern here. > For make it is often suggested to use slightly more threads than > available CPUs, when some threads are doing IO you still get to use all > CPUs. There was a "*2" ... I know, but same reasoning as above. And given that cpu_count is cores*threads with, typically, threads >= 2, this overcommittment is less important today. Jan -- Siemens AG, Corporate Technology, CT RDA IOT SES-DE Corporate Competence Center Embedded Linux