From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 6657041487194226688 X-Received: by 2002:a50:915d:: with SMTP id f29mr382708eda.12.1549965089227; Tue, 12 Feb 2019 01:51:29 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a50:cb4b:: with SMTP id h11ls3929157edi.5.gmail; Tue, 12 Feb 2019 01:51:28 -0800 (PST) X-Google-Smtp-Source: AHgI3IYD9tL1XN7BO19qW7U/1i9y5Z4ax/kTRtxD9nGwlzMOnZ3KYbI2cJgSTHDru6qhUS8jMObD X-Received: by 2002:aa7:c393:: with SMTP id k19mr384666edq.6.1549965088710; Tue, 12 Feb 2019 01:51:28 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1549965088; cv=none; d=google.com; s=arc-20160816; b=BaqR4ppyQLsiIHPuoc8qxGNYyr/N7uPXSaG4Vjrpp015bk9hLz9EwZc80Eu0jY3Lus cHGJH7LRHclM4B69RkQKUVbmGouuuqW53TcRANtP+yNCjXpRSsKgIUl3QAsMvy132/EW wB5/ZQC5AbsPEyzeIBhUyZTK71rj9Nuza19VFmRkvlatblIJarZ0gNcOt1WU12kAIQHm EgRl8N+P1H/wQE6Yu9LigrKxLzviiHgTz4RoZMWeQBpByh8Vk+0lpYtP32DbIutJ0Gg+ SlyG/Iv1d0M1SyTxi04/9Nzre0ogjzJeAbYn+OpmXkXnBRg+WACKGGwDjqzq0j+v4hvR jzqA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:content-language:in-reply-to:mime-version :user-agent:date:message-id:from:references:cc:to:subject; bh=qfAZz4RP7llw4wsn01C5yUyHrvJpbYO4FhHZ9f/TMco=; b=QcuJ68Zk061tzUzhs+/QMpWq6bry63TRBTP1T2SWr8ZE/uGRfQtj0EMq48QF0Z40eb hhygRATB7obQdmuerCyB/g/J7MZGouehTYK7SG6aZBZRIMhqQoKm1DC7ETnjVAN87/rE 2FaeAnXC65YZiyunY9GkKOWCBL6Vr9MeYgiu5ZPtUHOHKdkWdwSolYNYDWdKvEoQp9Lc KHdAiEnnNNCPg2CE4rtEbSXRQwKAwv9Z49tAp1viEhvHAe5VUcs2CdRL5EBOBDkJfd2a JTxnuM9xc2zIcp7hovr1YKKJCSNciaaJEI1QR3FfbotHs1yBVpVnyLumuKnJ29YTGh4Z h9RQ== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Return-Path: Received: from thoth.sbs.de (thoth.sbs.de. [192.35.17.2]) by gmr-mx.google.com with ESMTPS id a3si652177eda.0.2019.02.12.01.51.28 for (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 12 Feb 2019 01:51:28 -0800 (PST) Received-SPF: pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.2 as permitted sender) client-ip=192.35.17.2; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of claudius.heine.ext@siemens.com designates 192.35.17.2 as permitted sender) smtp.mailfrom=claudius.heine.ext@siemens.com Received: from mail1.sbs.de (mail1.sbs.de [192.129.41.35]) by thoth.sbs.de (8.15.2/8.15.2) with ESMTPS id x1C9pPJS008050 (version=TLSv1.2 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Tue, 12 Feb 2019 10:51:26 +0100 Received: from [139.25.69.181] (linux-ses-ext02.ppmd.siemens.net [139.25.69.181]) by mail1.sbs.de (8.15.2/8.15.2) with ESMTP id x1C9pQdv025947; Tue, 12 Feb 2019 10:51:26 +0100 Subject: Re: [PATCH 1/4] Remove all uses of subprocess.call(shell=True) To: Harald Seiler , isar-users@googlegroups.com Cc: Claudius Heine References: <20190212092046.6508-1-hws@denx.de> <20190212092046.6508-2-hws@denx.de> From: Claudius Heine Message-ID: <815e882f-71ba-7d8d-4628-db9c5bea6c9b@siemens.com> Date: Tue, 12 Feb 2019 10:51:26 +0100 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.4.0 MIME-Version: 1.0 In-Reply-To: <20190212092046.6508-2-hws@denx.de> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-TUID: cWwvLilOX4DM Hi Harald, On 12/02/2019 10.20, Harald Seiler wrote: > Using shell-expansion in subprocess calls has a lot of nasty > side-effects that are hard to debug should one ever surface. > This commit changes all uses of subprocess to > > A) not use shell=True by giving a list of args instead of > a command-line. > B) ensure return-codes are handled appropriately. > > Furthermore, in some places the subprocess usage was changed to > be more idiomatic. > > Co-authored-by: Claudius Heine > Signed-off-by: Harald Seiler > --- > meta/classes/base.bbclass | 10 ++++++---- > meta/classes/image.bbclass | 9 +++++---- > meta/classes/isar-bootstrap-helper.bbclass | 9 ++++----- > meta/classes/isar-events.bbclass | 13 +++++++------ > meta/classes/patch.bbclass | 16 ++++++++++++---- > 5 files changed, 34 insertions(+), 23 deletions(-) > > diff --git a/meta/classes/base.bbclass b/meta/classes/base.bbclass > index 6ca93bf..6fcf452 100644 > --- a/meta/classes/base.bbclass > +++ b/meta/classes/base.bbclass > @@ -170,15 +170,17 @@ addtask clean > do_clean[nostamp] = "1" > python do_clean() { > import subprocess > + import glob > > for f in (d.getVar('CLEANFUNCS', True) or '').split(): > bb.build.exec_func(f, d) > > - dir = d.expand("${WORKDIR}") > - subprocess.call('sudo rm -rf ' + dir, shell=True) > + workdir = d.expand("${WORKDIR}") > + subprocess.check_call(["sudo", "rm", "-rf", workdir]) > > - dir = "%s.*" % bb.data.expand(d.getVar('STAMP', False), d) > - subprocess.call('sudo rm -rf ' + dir, shell=True) > + stamppath = bb.data.expand(d.getVar('STAMP', False), d) > + stampdirs = glob.glob(stamppath + ".*") > + subprocess.check_call(["sudo", "rm", "-rf"] + stampdirs) > } > > # Derived from OpenEmbedded Core: meta/classes/base.bbclass > diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass > index be00817..930c245 100644 > --- a/meta/classes/image.bbclass > +++ b/meta/classes/image.bbclass > @@ -27,16 +27,17 @@ def get_image_name(d, name_link): > full = d.getVar("IMAGE_FULLNAME", True) + "." + base > return [base, full] > if os.path.islink(path_link): > - return get_image_name(d, os.path.relpath(os.path.realpath(path_link), > - '/')) > + return get_image_name(d, os.path.relpath(os.path.realpath(path_link), '/')) > + > return ["", ""] > > def get_rootfs_size(d): > import subprocess > rootfs_extra = int(d.getVar("ROOTFS_EXTRA", True)) > > - output = subprocess.check_output(['sudo', 'du', '-s', '--block-size=1k', > - d.getVar("IMAGE_ROOTFS", True)]) > + output = subprocess.check_output( > + ['sudo', 'du', '-s', '--block-size=1k', d.getVar("IMAGE_ROOTFS", True)] > + ) > base_size = int(output.split()[0]) > > return base_size + rootfs_extra * 1024 > diff --git a/meta/classes/isar-bootstrap-helper.bbclass b/meta/classes/isar-bootstrap-helper.bbclass > index d780b85..3e11098 100644 > --- a/meta/classes/isar-bootstrap-helper.bbclass > +++ b/meta/classes/isar-bootstrap-helper.bbclass > @@ -9,11 +9,10 @@ IMAGE_TRANSIENT_PACKAGES ??= "" > > def get_deb_host_arch(): > import subprocess > - host_arch = subprocess.Popen("dpkg --print-architecture", > - shell=True, > - env=os.environ, > - stdout=subprocess.PIPE > - ).stdout.read().decode('utf-8').strip() > + host_arch = subprocess.check_output( > + ["dpkg", "--print-architecture"], > + encoding="utf-8", utf-8 should be the default encoding, or is it? I would remove that if it is. Other check_output calls don't have that. > + ).strip() > return host_arch > > #Debian Distribution for SDK host > diff --git a/meta/classes/isar-events.bbclass b/meta/classes/isar-events.bbclass > index c4a6149..a178d05 100644 > --- a/meta/classes/isar-events.bbclass > +++ b/meta/classes/isar-events.bbclass > @@ -16,13 +16,14 @@ python isar_handler() { > > basepath = tmpdir + '/work/' > > - with open(os.devnull, 'w') as devnull: > - with open('/proc/mounts', 'rU') as f: > - lines = f.readlines() > - for line in lines: > + with open('/proc/mounts') as f: > + for line in f.readlines(): > if basepath in line: > - subprocess.call('sudo umount -l ' + line.split()[1], > - stdout=devnull, stderr=devnull, shell=True) > + subprocess.call( > + ["sudo", "umount", "-l", line.split()[1]], > + stdout=subprocess.DEVNULL, > + stderr=subprocess.DEVNULL, > + ) > } > > isar_handler[eventmask] = "bb.runqueue.runQueueExitWait bb.event.BuildCompleted" > diff --git a/meta/classes/patch.bbclass b/meta/classes/patch.bbclass > index 0bc449f..9d4441d 100644 > --- a/meta/classes/patch.bbclass > +++ b/meta/classes/patch.bbclass > @@ -21,10 +21,18 @@ python do_patch() { > > striplevel = fetcher.ud[src_uri].parm.get("striplevel") or "1" > > - cmd = "patch --no-backup-if-mismatch -p " + striplevel + \ > - " --directory " + src_dir + " --input " + workdir + basename > - bb.note(cmd) > - if subprocess.call(cmd, shell=True) != 0: > + cmd = [ > + "patch", > + "--no-backup-if-mismatch", > + "-p", > + striplevel, > + "--directory", > + src_dir, > + "--input", > + workdir + basename, > + ] IMO formatting could be improved there. I don't know about you, but I can read: cmd = [ "patch", "--no-backup-if-mismatch", "-p", striplevel, "--directory", src_dir, "--input", workdir + basename, ] better. :) But that is just opinion, so if you differ I will not insist on it. Otherwise great work! Claudius > + bb.note(" ".join(cmd)) > + if subprocess.call(cmd) != 0: > bb.fatal("patching failed") > except bb.fetch2.BBFetchException as e: > raise bb.build.FuncFailed(e) > -- DENX Software Engineering GmbH, Managing Director: Wolfgang Denk HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany Phone: (+49)-8142-66989-54 Fax: (+49)-8142-66989-80 Email: ch@denx.de